Kyso
Search…
⌃K
Links

Github

Learn how to setup Github Actions so publishing your data science work becomes automatic!!

Prerequisites

  • Have a Kyso account - either on kyso.io or on your company's private Kyso installation.
  • Create a Kyso access token - follow these instructions. Save this for later!
  • Ensure your directory (or folders) contains a valid kyso.yaml file. Check the following instructions for more info.

How to publish to Kyso with Git Actions

About the kyso-push step variables:
  • Theusernameand token fields take their value from secret variables, to make the system work the user has to create the kyso auth token and define the KYSO_USERNAME and KYSO_TOKEN as explained on the github documentation.
  • Theurl has to point to your Kyso deployment. So if your company, Acme Inc. has its own Kyso instance available on https://acme.kyso.io, that is the value that has to be assigned to it.
Here is an example Github repository that you can clone and edit to test out the publishing flow: https://github.com/KyleOS/kyso-demo
  1. 1.
    Create a .github/workflows directory in your repository on GitHub if this directory does not already exist.
  2. 2.
    In the .github/workflows directory, create a file named kyso-action.yml.
  3. 3.
    Copy the following YAML contents into the kyso-action.yml file:
name: 'Push To Kyso'
on:
push:
tags:
- '*'
jobs:
push-to-kyso:
runs-on: ubuntu-latest
name: Job to push a report to kyso
steps:
- name: Clone repo step
uses: actions/[email protected]
- name: Kyso push step
id: kyso-push
uses: kyso-io/push-[email protected]
with:
username: ${{ secrets.KYSO_USERNAME }}
token: ${{ secrets.KYSO_TOKEN }}
url: 'https://kyso.io/'
This means that on each new push to a tag on the repository, a Github action will be generated to run the commands in our action file above.
Remember to also ensure that your metadata specs in the sub-directory YAML files, are correct:
Note that both repositories contain multiple sub-folders and so take on the type: meta , and each sub-folder will have it's own kyso.yaml file with specifications on that individual project's metadata (e.g. title, description, type, tags, etc.)
Note that the example repo contains multiple sub-folders and so take on the type: meta, and each sub-folder will have it's own kyso.yaml file with specifications on that individual project's metadata (e.g. title, description, type, tags, etc.)

Executing the CI/CD Pipeline

When we commit our work to Github, an Action will queued to execute. If we navigate to https://github.com/KyleOS/kyso-demo/actions we'll see that our runs are executing, first by installing the Kyso CLI, logging the user in, and then publishing the report.
If we check the logs, this is what we will see:
$ npm install -g kyso
added 234 packages, and audited 235 packages in 19s
33 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities
$ kyso login --kysoInstallUrl https://kyso.io --provider kyso --username [YOUR USERNAME] --token [YOUR TOKEN]
Logged successfully
$ kyso push
Uploading report '.'
Founded 4 files:
Processing .github/workflows/kyso-action.yml
Processing README.md
Processing index.html
Processing kyso.yaml
Uploading files. Wait a moment..
🎉🎉🎉 Reports were uploaded
https://kyso.io/kyso-examples/life-sciences/
🎉🎉🎉
Now we can navigate to Kyso at https://kyso.io/kyso-examples/life-sciences/ to see the published list of our example reports!

Maintaining a QA process with Pull Requests

By integrating Kyso into Git actions, the workflow is super flexible and you can refine this further and select if you want to integrate a specific branch, tag, or whatever else fits your specific workflow needs!
Here is a link to all the different event types that trigger these workflows:

Publishing only when a PR is merged into the main branch

So, for example, if you work with a team of scientists, data scientists, etc.., all pushing and pulling to and from the same repository, you're going to want to control how, when and what changes get published to Kyso.
Now there is ongoing debate on how to do this. For example, see this discussion:
There are a couple of different commands we can use to ensure that only when a PR is merged into our main branch is our workflow triggered. However, we can actually just use our existing action file:
name: Push To Kyso
on:
push:
branches:
- main
jobs:
Push-To-Kyso:
runs-on: ubuntu-latest
steps:
- uses: actions/[email protected]
- uses: actions/[email protected]
with:
node-version: 18
- run: npm install -g kyso
- run: kyso login --kysoInstallUrl https://kyso.io --provider kyso --username [YOUR USERNAME] --token [YOUR ACCESS TOKEN]
- run: kyso push
Because a merged pull request always results in a push, we can just use the push event to accomplish our goal. So the above workflow will run when a PR is merged or when a commit is made directly to the master branch.
To make this workflow even more secure, it has been recommended that you add branch protection rules to your main branch: