Table of Contents
What are GitHub Actions?
Recently I created Portainer App Templates for LinuxServer.io Docker containers and Docker scripts for LinuxServer.io Docker containers.
Since I was looking for a way to run the Python scripts periodically and publishing the output to GitHub, I thought of taking a look at GitHub Actions.
From the GitHub Actions website.
GitHub Actions makes it easy to automate all your software workflows, now with world-class CI/CD. Build, test, and deploy your code right from GitHub. Make code reviews, branch management, and issue triaging work the way you want.
Sounds pretty good right?
It turns out that GitHub Actions are very well suited for what I’m looking for. And fairly easy to set up and maintain.
Let’s see how this works.
Creating the workflow
Set up the workflow
From the repository on GitHub, navigate to the Actions tab.
Select the following workflow from “Continuous integration workflows” as a starting point.
Choose “Set up this workflow”.
Detailed information about Building and testing Python can be found here.
Change the name from python-app.yml to generate_templates.yml and commit the changes.
You now have a YAML file generate_templates.yml in .github/workflows that looks like this.
name: Python application
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.9
uses: actions/setup-python@v2
with:
python-version: 3.9
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install flake8 pytest
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Test with pytest
run: |
pytestCode language: YAML (yaml)This means that on every push or pull_request a job is executed that consists of a number of steps.
Because we pushed a new file, the workflow will run immediately. This is done by the runner. The runner is the application that runs a job from a workflow. It is used by GitHub Actions in the hosted virtual environments, or you can self-host the runner in your own environment.
If we go to the Action tab, we can see the workflow run “Create generate_templates.yml” with a red cross in front of it indicating it failed.
When we select the workflow run and expand the job named “build”, we see that it failed in step “Test with pytest” with exit code 5.
Exit code 5 means that “No tests were collected”. Understandable, because we did not upload or run any Python code, let alone tests.
On schedule
Let have a look at the generate_templates.yml file again.
name: Python application
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.9
uses: actions/setup-python@v2
with:
python-version: 3.9
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install flake8 pytest
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Test with pytest
run: |
pytestCode language: YAML (yaml)We don’t need to run the workflow on every push or pull_request. Instead, we need to run the workflow on a certain schedule.
For this we can use the following code.
on:
schedule:
- cron: "* * * * *"Code language: YAML (yaml)Where “* * * * *” is a cron expression:
# * * * * *
# ┬ ┬ ┬ ┬ ┬
# │ │ │ │ │
# │ │ │ │ └────► day of the week (0 - 6)
# │ │ │ └─────────► month (1 - 12)
# │ │ └──────────────► day of the month (1 - 31)
# │ └──────────────────► hour (0 - 23)
# └───────────────────────► minutes (0 - 59)
Code language: Markdown (markdown)Because we want to run the program every day at 00:00 the code is changed to.
on:
schedule:
- cron: "0 0 * * *"Code language: YAML (yaml)Execute Python script
We can also delete the following step since there are no tests to be run.
- name: Test with pytest
run: |
pytestCode language: YAML (yaml)Instead, we need a step to execute the Python script.
- name: Execute Python script
run: |
python generate_templates.pyCode language: YAML (yaml)While we’re at it, change the name of the workflow into Generate templates as well.
The workflow now looks like this.
name: Generate templates
on:
schedule:
- cron: "0 0 * * *"
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.9
uses: actions/setup-python@v2
with:
python-version: 3.9
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install flake8 pytest
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Execute Python script
run: |
python generate_templates.pyCode language: YAML (yaml)The output is now created on the server where the job is executed. So now we need to push the data to another repository or else it will be deleted when the workflow end.
Push a file to another repository
The script generate_templates.py that is used in Portainer App Templates for LinuxServer.io Docker containers generates a file. So we need to find a way to push that file to the destination repository.
For this we can search the GitHub Marketplace for GitHub Actions.
The GitHub Action that we will use is Push a file to another repository.
- name: Pushes output to destination repository
uses: dmnemec/copy_file_to_another_repo_action@main
env:
API_TOKEN_GITHUB: ${{ secrets.API_TOKEN_GITHUB }}
with:
source_file: 'templates-2.0.json'
destination_repo: 'technorabilia/portainer-templates'
destination_folder: 'lsio/templates'
user_email: 'simon@technorabilia.com'
user_name: 'technorabilia'
commit_message: 'Workflow update'Code language: YAML (yaml)You need to set the API_TOKEN_GITHUB in the GitHub Settings, Developer settings, Personal access tokens (set the repo scope). After that you need to add the token to the repository where the workflow is run. For this go to Settings, Secrets and add the repository secret.
Add the step to the generate_templates.yml file and it will look something like this.
name: Generate templates
on:
schedule:
- cron: "0 0 * * *"
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.9
uses: actions/setup-python@v2
with:
python-version: 3.9
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install flake8 pytest
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- name: Lint with flake8
run: |
# stop the build if there are Python syntax errors or undefined names
flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Execute Python script
run: |
python generate_templates.py
- name: Pushes output to destination repository
uses: dmnemec/copy_file_to_another_repo_action@main
env:
API_TOKEN_GITHUB: ${{ secrets.API_TOKEN_GITHUB }}
with:
source_file: 'templates-2.0.json'
destination_repo: 'technorabilia/portainer-templates'
destination_folder: 'lsio/templates'
user_email: 'simon@technorabilia.com'
user_name: 'technorabilia'
commit_message: 'Workflow update'Code language: YAML (yaml)Push a folder to another repository
The script generate_scripts.py that is used in Docker scripts for LinuxServer.io Docker containers generates a folder. So we cannot use the Push a file to another repository GitHub Action for this. Instead we will use the Push a folder to another repository.
Create a new workflow. Change the name from python-app.yml to generate_scripts.yml and commit the changes.
The workflow will be the same as for the one we created earlier, except for the last step.
In this case, the last step will look like.
- name: Pushes output to destination repository
uses: crykn/copy_folder_to_another_repo_action@v1.0.6
env:
API_TOKEN_GITHUB: ${{ secrets.API_TOKEN_GITHUB }}
with:
source_folder: 'lsio'
destination_repo: 'technorabilia/docker-bits'
destination_folder: 'lsio'
user_email: 'simon@technorabilia.com'
user_name: 'technorabilia'
destination_branch: 'main'
commit_msg: 'Workflow update'Code language: YAML (yaml)Viewing the workflow results
Once the workflow has run, we can have a look at the results of the workflow.
If we go to the Action tab, we can see the workflow run “Generate templates” (or “Generate scripts”) with a green check mark in front which means the workflow completed succesfully.
We can also see the step “Pushes output to destination directory” ran succesfully. So we should see the output in the destionation repository.
Note the running time for each step. This will be deducted from the “running time” you have left for this month.
If an error occurs, a e-mail notification is send. Otherwise we can assume that the workflow will continue to run according to schedule!
Considerations
If we go to GitHub Settings, Developer settings, Billing & plans you can look at your currect plan and usage for this month.
Currently, GitHub Free included 2000 minutes of “running time” per month.