Generating Hugo Posts From an API Using Python

  • Sat, Nov 14, 2020
  • 4-minute read

For my personal side project travelhackingtool.com, which is an aviation database and API, containing thousands of airports, I was looking for an easy way and automated to create Hugo Posts for my airports based on the data of the API. If you are curious which airports are available on the site then feel free to browse this page.

Travelhackingtool’s API which can be consumed through Rapid API for free, currently provides the airports in a JSON format like this:

1
2
3
4
5
6
7
8
9

{
"city_code":"MUC"
"code":"MUC"
"country":"DE"
"latitude":48.353004
"longitude":11.790143
"name":"Franz Josef Strauss Airport"
"time_zone":"Europe/Berlin"
}

What I ultimately was looking for was a visually appealing HTML page like this one based on the data provided in the JSON format to provide some value for less techy visitors. Notice how the data in the post precisely matches the data from the API?

MUC Airport on travelhackingtool.com

How we can generate the posts in theory

Let’s have a look at how a process for generating posts from an API could work in theory.

It works like this:

  1. Fetch an up-to-date list of all airports from my API using Python requests
  2. For each airport in the API response:
    1. Fill out a generic template using jinja2
    2. Save template as a markdown file
  3. Generate static sites using Hugo

What the implementation looks like in Python

Once I knew how it should work in theory, it was relatively easy to implement it in Python. I started off by creating a markdown template named template.md for the airports. Notice the placeholders such as {{airport.code}}, which will later on be filled out by the Python script for each of the many thousand airports.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

+++
title = "{{airport.name}} ({{airport.code}})"
image = ""
author = "Admin"
date = 2019-11-07T05:00:00Z
description = "{{airport.name}} ({{airport.code}}) | Provided by www.travelhackingtool.com the only free aviation data API."
categories = ["Airport"]
type = "airport"
draft = false
latitude = {{airport.latitude}}
longitude = {{airport.longitude}}
zoom = 5
+++

{{airport.name}} ({{airport.code}}) is an airport located at the decimal coordinates `{{airport.latitude}},{{airport.longitude}}`. {{airport.name}} Airport's timezone is {{airport.time_zone}}. 

## IATA Code

{{airport.code}}

## City Code

{{airport.city_code}}

## Time Zone

{{airport.time_zone}}

The Python script for filling out the airports looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

import requests
from jinja2 import Template
import codecs
import time, os
base_url = "http://<url to API>"
headers = {
    'Authorization': '<API Token>', 'Content-type': 'application/json'
}

# API call for fetching the airports
r = requests.get(base_url, headers=headers).json()
curpath = os.path.abspath(os.curdir)

for airport in r:
    # Open Template and populate it
    with open('template.md', 'r') as file:
        template = Template(file.read(),trim_blocks=True)
    rendered_file = template.render(airport=airport)

    # write the markdown file
    output_file = codecs.open(os.path.join(curpath, "md","{}-Airport.md".format(airport["code"])), "a", "utf-8")
    output_file.write(rendered_file)
    output_file.close()

How the generated Markdown file looks

The generated output will look like this, which Hugo will be able to turn into a nice looking static website.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

+++
title = "Franz Josef Strauss Airport (MUC)"
image = ""
author = "Admin"
date = 2019-11-07T05:00:00Z
description = "Franz Josef Strauss Airport (MUC) | Provided by www.travelhackingtool.com the only free aviation data API."
categories = ["Airport"]
type = "airport"
draft = false
latitude = 48.353004
longitude = 11.790143
zoom = 5
+++

Franz Josef Strauss Airport (MUC) is an airport located at the decimal coordinates `48.353004,11.790143`. Franz Josef Strauss Airport Airport's timezone is Europe/Berlin. 

## IATA Code

MUC

## City Code

MUC

## Time Zone

Europe/Berlin

All that is left is to copy the generated Markdown file into your corresponding content directory and run a hugo command on it. The generate process, might take a wee bit of time, depending on how many markdown files, Hugo has to process.

Summary

As you can see, it is effortless to generate a Hugo Post from an API, by merely using Pythons web templating technique jinja2 . I hope my script is also useful for your own Hugo automation project.

Travel Hacking pythonrequestswebmarkdownJSONAPIHugo