Bitbucket
Learn how to setup CI pipelines so publishing your data science work becomes automatic!!
- Have a Kyso account - either on kyso.io or on your company's private Kyso installation.
- Ensure your directory (or folders) contains a valid
kyso.yamlfile. Check the following instructions for more info.
About the
kyso-push step variables:- The
usernameandtokenfields take their value from secret variables, to make the system work the user has to create the kyso auth token and define theKYSO_USERNAMEandKYSO_TOKENas explained on the github documentation. - Create your Access Token according to this document: https://docs.kyso.io/posting-to-kyso/kyso-command-line-tool/authorization#access-token
- The
urlhas to point to your Kyso deployment. So if your company, Acme Inc. has its own Kyso instance available on https://acme.kyso.io, that is the value that has to be assigned to it.
Here is an example Bitbucket repo (with a simple Jupyter notebook project):
Bitbucket
Bitbucket
Bitbucket Demo Repository
In the root directory of the repository, we have created a
bitbucket-pipelines.yml that details the actions we want to run and when:image: node:18
pipelines:
branches:
main:
- step:
name: 'Publishing to Kyso'
trigger: 'automatic'
script:
- npm install -g kyso
- kyso login --kysoInstallUrl https://kyso.io --provider kyso --username [YOUR EMAIL] --token [YOUR ACCESS TOKEN]
- kyso push
Note that `--kysoInstallUrl` needs to point to your company's Kyso deployment. So if your company, Acme Inc., has their Kyso on https://acme.kyso.io, then that is your install url address!
So now every time we make a commit to this repository, whether on the UI or from the Command Line, the above script will be run, publishing and updating our work in Kyso.
Note that this is a super simple example of some Jupyter notebooks and using the example Starter pipeline configuration, but this can be integrated into much more powerful pipeline flows.
For example, you can decide to only push specific branches or tags.
Don't see or can't figure out how to set up your desired workflow? Simply reach out to us, let us know your use case and how you want your pipeline to work, and we'll send you the documentation!!
Using our example repository from above, as we commit changes to the Bitbucket project, a Docker image is created to run our job. We can check out the pipeline logs at https://bitbucket.org/KyleOS/kyso-demo/pipelines. Clicking into one of the most recent successful merges, we should see something like this:
Bitbucket Pipeline
Similar to our documentation on Github Actions, we can maintain a QA process, ensuring only merged/reviewed work is pushed upstream to Kyso.
So, for example, if you work with a team of scientists, data scientists, etc.., all pushing and pulling to and from the same repository, you're going to want to control how, when and what changes get published to Kyso.
Because a merged pull request always results in a push, we can just use the push event to accomplish our goal. So the above workflow will run when a PR is merged or when a commit is made directly to the master branch.
To make this workflow even more secure, it has been recommended that you add branch protection rules to your main branch!
Last modified 2mo ago