# What is Kyso? Welcome to the official Kyso documentation 👋 Kyso is your company's universal data portal for all technical and non-technical reporting and collaboration. We built Kyso so teams and organisations could have a single reference point for all data-based reporting and collaboration, a platform that connects the technical - data scientists, engineers, biostatisticians, etc. - with the non-technical - everyone else around the business! Kyso integrates seamlessly into technical workflows so publishing & keeping research up to date becomes automatic. Your colleagues don't have to waste any more time searching for, or requesting access to, research relevant to their role in the company; they can now just search & discover on-demand. Collaboration also happens directly on the reports with Google Doc-style commenting. ## Kyso in Action The video demo below takes you through the new Kyso platform, which has now been open sourced, so companies can self-host the product. Please tag us in an issue on the repository if you have any questions! {% embed url="" %} The Kyso Platform {% endembed %} ## Table of Contents ### 1. Getting Started {% content-ref url="/pages/7mbJZNPkMh0XucNdQAtT" %} [Architecture and Deployment](/getting-started/architecture-and-deployment) {% endcontent-ref %} There are 3 main feature-sets to the Kyso Platform: ### 1. Publishing and Sharing {% content-ref url="/pages/-LjzhPN6Oo\_n\_4fh9D3f" %} [Creating Reports In-App](/posting-to-kyso/creating-reports-on-kyso) {% endcontent-ref %} {% content-ref url="/pages/cA7WA2QbPGWMRdols7Gv" %} [Kyso's Command Line Tool](/posting-to-kyso/kysos-command-line-tool) {% endcontent-ref %} {% content-ref url="/pages/-Ljzh\_jD9kgQyhSj9kYb" %} [Integrating with Git](/posting-to-kyso/connect-a-github-repo-to-kyso) {% endcontent-ref %} ### 2. Search and Discovery {% content-ref url="/pages/XUEO9Mcw9GGcUNZUkpIZ" %} [Searching Reports](/search-and-discovery/searching-reports) {% endcontent-ref %} {% content-ref url="/pages/gxCQdE3Sb6KJcSC1fKct" %} [Browsing Files & Versions](/search-and-discovery/browsing-files-and-versions) {% endcontent-ref %} ### 3. Collaboration {% content-ref url="/pages/coyft8LdIUc7tE0EZ3tS" %} [Report Comments](/collaboration/report-comments) {% endcontent-ref %} {% content-ref url="/pages/33OPbCI57TntRqnmb6C3" %} [Report Tasks](/collaboration/report-tasks) {% endcontent-ref %} {% content-ref url="/pages/JzXpwz7n4o37qLnJcoMK" %} [Business Notifications](/collaboration/business-notifications) {% endcontent-ref %} # Architecture and Deployment Kyso can be installed on private servers or cloud instance, keeping your company's knowledge secure. ## Components **Kyso** is a Web Application organized as a set of services and applications packaged in containers prepared to be deployed and work together using a `kubernetes` cluster (the components could be deployed using other orchestration systems if required, but our current deployment system is focused on `k8s`).

Kyso Architecture

The two main components of the system are: * `kyso-api`, `NodeJS` server that implements the backend functionality using internal and external services and provides a REST API to access it. * `kyso-front`, the system's web user interface, developed using the `NextJS` framework, does its work calling the relevant API methods on behalf of the user. Other support services used are: * `kyso-scs`, module used to manage the reports storage and publication; it includes four sub-components that work together: * an internal ssh server to manage report files (it is accessed only by the API, users don't have direct access to it). * a web server that provides access to the files with access control (the access validation is managed by calling the API). * a Java application used to index the content of the reports and add the relevant information to the search engine database. * a Web server that allows us to run scripts with direct access to the filesystem for maintenance tasks and information retrieval. * `elasticsearch`, search engine used to search and index comments, discussions and reports uploaded to the platform. * `mongodb`, a database used to keep all the data related to the platform that is not inside the reports (i.e. configuration and user data, discussions and comments, etc.) * `nats`, a message oriented *middleware* used by the API to emit messages related to different events that can be consumed by specialised micro-services * `onlyoffice-ds`, online editor for text documents, spreadsheets, presentations, forms and PDF reader used to display office formats from the frontend Additionally the platform supports the deployment of *optional* message consumers (`nats` clients) that perform specialised tasks (send notifications using email or slack, display an activity feed, etc.) ## Deployment system Right now the recommended way of deploying Kyso is to use `kitt` (*Kyso Internal Tool of Tools*), a command line tool that includes support to configure and deploy third party components (i.e. `mongodb`, `elasticsearch`, …​) and our own services (`kyso-api`, `kyso-front`, `kyso-scs`, …​) on kubernetes clusters using `helm` and `kubectl`. The tool is also capable of installing a cluster for testing purposes on a single Linux server (using `k3d`) and install and configure additional services on it or an existing kubernetes cluster. # Jupyter Notebooks Create your narrative and document results right alongside your code and graphics, Kyso will render everything as a beautiful data document. ## Publishing Notebooks Kyso renders Jupyter notebooks as beautiful blog posts, while maintaining any interactivity in your graphical outputs. Check out the video below to see how notebooks render on Kyso & an overview of the collaboration features available on these report types: {% embed url="" %} Jupyter Notebooks on Kyso {% endembed %} Any plotting library that uses a Javascript kernel in the backend will render nicely on Kyso. So any graph you make in Jupyter, using Plotly, Bokeh, Altair, etc.. will work. {% content-ref url="/pages/-Ma3Ll39WU4geL8NFSba" %} [Bokeh Plots and Kyso](/kysos-renderer/jupyter-faq/bokeh-plots-and-kyso) {% endcontent-ref %} Note that graphs & widgets using a python kernel are currently not supported as notebook-rendered outputs. This feature is in Kyso's roadmap however. For now you can publish these types of widgets as HTML, and they will work: {% content-ref url="/pages/qRkNxsUCIee3Clo06K1T" %} [HTML](/kysos-renderer/html) {% endcontent-ref %} ## Configuration {% hint style="info" %} **Kyso now has Google Doc-style inline commenting on Jupyter notebooks!** {% endhint %} This was made possible by Jupyter [adding unique cell ids to notebooks](https://jupyter.org/enhancement-proposals/62-cell-id/cell-id.html). Kyso leverages these ids (which are unchanging) to create comments that refer to specific cells in the report. However, only Jupyter notebooks of v4.5 and above will have cell ids by default. If you want to upload existing notebooks of older versions without updating everything locally, you can optionally allow Kyso to apply cell ids to your notebooks. Make sure you have installed our CLI to the latest version: {% content-ref url="/pages/cA7WA2QbPGWMRdols7Gv" %} [Kyso's Command Line Tool](/posting-to-kyso/kysos-command-line-tool) {% endcontent-ref %} And run the following from the command line, in the directory with the target notebooks: ```bash // kyso format --jupyter ``` This will prompt you with: ```bash Jupyter notebooks of v4.5 & above have unique cell identifiers, allowing Kyso to add inline comments to the reports. It seems that you are using an older version of Jupyter. If you want to allow for inline comments on your report without updating your version of Jupyter, select 'yes' and Kyso will process all notebooks in this push & set a random identifier automatically to all cells, with no side effects to the content of the report(s). If you select 'no' the notebooks will be published without cell ids. These changes will modify your .ipynb files in your local filesystem, do you want to continue? (y/n) ``` ## Collaboration ### Inline Commenting & Tasks Users can comment directly on a specific cell on Kyso now - whether that be code input or a graphical output, just as they would on Google Docs or Notion.
{% hint style="info" %} You can even assign inline comments "status", thus creating a project task. {% endhint %} ### Cell Referencing By leveraging cell ids in notebooks, we've also built **Report Snippets**. This is especially useful when sharing notebook reports that contain lots of graphs, and you want the receiver to hone in on a specific section of the report. Simply click on the **link** icon next to a cell to open up that snippet in a separate page. **Note the URL in the image below:**
Users can open up the rest of the notebook in the top right of the report! ## Jupyter Diffing Users can compare changes across report versions on Jupyter-based reports, just as one would do so on Github. On the report page, follow the link shown below:

Compare Report Versions

On the next page, select the report versions you want to compare. After a moment, the report will be rendered below and you will see changes from one version to the next highlighted, just like on Github!

View Changes

# Jupyter FAQ Troubleshooting Publishing & Rendering Problems {% content-ref url="/pages/-LzD-YQmi9Yx89dNXDrX" %} [Kyso's Jupyter Renderer](/kysos-renderer/jupyter-faq/kysos-jupyter-renderer-faq) {% endcontent-ref %} {% content-ref url="/pages/-Ma3Ll39WU4geL8NFSba" %} [Bokeh Plots and Kyso](/kysos-renderer/jupyter-faq/bokeh-plots-and-kyso) {% endcontent-ref %} # Kyso's Jupyter Renderer This article lays out some important points to know about how Jupyter notebooks are rendered on Kyso. ## Images We don't support image links with spaces in the name. So if you link an image in a markdown cell do it like this: ``` ![alt text](image_name.png) ``` and don't do it like this: ``` ![alt text](image name.png) ``` ## IPyWidgets While interactive javascript plots like Plotly and Bokeh do work (as well as associated client side widgets), IPyWidgets won't work as they require a running kernel - which Kyso does not provide (**yet!!**). You could alternatively export these widgets to HTML, which Kyso does support: {% content-ref url="/pages/qRkNxsUCIee3Clo06K1T" %} [HTML](/kysos-renderer/html) {% endcontent-ref %} # Bokeh Plots and Kyso Use this snippet to make bokeh plots work seamlessly in Jupyter and Kyso. To make bokeh plots work inside Jupyter and render on Kyso, we need to save the plot to a virtual html file and then show it in the notebook: ``` from bokeh.plotting import figure, output_file, save from IPython.display import IFrame from IPython.core.display import display, HTML import tempfile def bokeh_show(plot): tmp_output_filename = tempfile.NamedTemporaryFile(suffix='.html').name output_file(tmp_output_filename) save(plot) f = open(tmp_output_filename, "r") display(HTML(f.read())) ``` Then we can use this function `bokeh_show` to show any plots we create: ``` p = figure(title="Bokeh test", plot_width=300, plot_height=300) p.circle([1, 2], [3, 4]) bokeh_show(p) ``` \[1] \[2] # HTML Publish amazing graphics for discovery that otherwise get shared once & then go unused sitting on random local file servers around the business! You can easily share HTML files on Kyso, where they are indexed for search and discovery. There are two kinds of HTML-based reports to consider. ## Single-page HTML Publishing HTML files is pretty straightforward. Simply configure your report's metadata, setting the HTML file as the "main" variable in the YAML header, and then publish! You can do this with the CLI tool OR simply upload the files in the browser: {% content-ref url="/pages/-LjzhPN6Oo\_n\_4fh9D3f" %} [Creating Reports In-App](/posting-to-kyso/creating-reports-on-kyso) {% endcontent-ref %} **Note that you can "Open in Full Screen" on these report files to get that full HTML experience in the browser!**

Open HTML in full screen!

{% hint style="danger" %} If possible, don't edit the HTML file name, since this can impact the styling of the rendered report (example border overflow). HTML reports on Kyso are simply embedded iframes. {% endhint %} ## Full Websites There is a little more to publishing full websites over simple, single-page HTML files. For this doc, we'll use Jupyter Books as an example. But do note that Kyso will render any website. {% hint style="info" %} Since websites entail publishing not just one file, but many, potentially with lots of other accompanying files & sub-folders, users should publish these to Kyso using the CLI tool. {% endhint %} ## 1. Jupyter Books A Jupyter Book is compiled to HTML from multiple Jupyter Notebooks, with optional JS and/or CSS styling. See Jupyter's gallery for examples: ​ We cover the basics of Jupyter Book creation in this document. Please refer to the official Jupyter documentation for more robust information on creating your Book reports: ### Installation First you need to install the jupyter-book package. You can install Jupyter Book via `pip`: ``` // pip install -U jupyter-book ``` or via [`conda-forge`](https://conda-forge.org/): ``` // conda install -c conda-forge jupyter-book ``` This will install everything you need to build a Jupyter Book locally. ### Build your book's HTML Once you’ve added content (notebooks) and configured your book, it’s time to build outputs for your book. You can use the `jupyter-book build` command line tool for this. You should already have a collection of notebook/Markdown files in your `mybookname/` folder, a `_toc.yml` file that defines the structure of your book, and any configuration you’d like in the `_config.yml` file. Now that your book’s content is in your book folder and you’ve defined your book’s structure in `_toc.yml`, you can build the HTML for your book. Do so by running the following command: ``` // jupyter-book build mybookname/ ``` This will generate a fully-functioning HTML site using a **static site generator**. The site will be placed in the `_build/html` folder, something like this: ``` mybookname └──_build └── html ├── _images ├── _static ├── index.html ├── intro.html ... ``` You can now publish to Kyso. Make sure you specify the main file in the reports YAML, for example: ``` main: _build/html/index.html ``` # Markdown Creating markdown reports on Kyso ## Pushing Existing Files Users can upload their markdown (.md) files to Kyso directly on the UI: {% content-ref url="/pages/-LjzhPN6Oo\_n\_4fh9D3f" %} [Creating Reports In-App](/posting-to-kyso/creating-reports-on-kyso) {% endcontent-ref %} or with the Kyso CLI tool: {% content-ref url="/pages/LEia9hOEUZaGfGYFxZbc" %} [Publishing & Downloading](/posting-to-kyso/kysos-command-line-tool/publishing-and-downloading) {% endcontent-ref %} ## Writing Reports from Scratch Kyso also has it's very own built-in markdown editor. On the Organisation or Channel dashboard, simply navigate to "**Post a report"** at the top and select **"Create new report in the UI".** This will take you to `https://kyso.io//create-report/`

Create a Report

Here you can write up a report from scratch, create new files & folders on the UI, and upload from your local machine. You can even build cool engineering process diagrams like [Mermaid](https://mermaid.js.org/) in the editor! # PDFs How to publish your PDF files to Kyso! ## OnlyOffice Kyso integrates with OnlyOffice, a free software office suite and ecosystem of collaborative applications that allows the platform to read & render various document types, including PDFs.

PDF Document on Kyso

## Publishing Pushing PDFs works like any other report type on Kyso. You can upload and categorise your PDF files any which way you prefer: {% content-ref url="/pages/-LjzhPN6Oo\_n\_4fh9D3f" %} [Creating Reports In-App](/posting-to-kyso/creating-reports-on-kyso) {% endcontent-ref %} {% content-ref url="/pages/cA7WA2QbPGWMRdols7Gv" %} [Kyso's Command Line Tool](/posting-to-kyso/kysos-command-line-tool) {% endcontent-ref %} {% content-ref url="/pages/JDCMDT073Dko60xXvGTj" %} [Importing Files from S3](/posting-to-kyso/importing-files-from-s3) {% endcontent-ref %} # PowerPoints How to publish your PowerPoint files to Kyso! ## OnlyOffice Kyso integrates with OnlyOffice, a free software office suite and ecosystem of collaborative applications that allows the platform to read & render various document types, including PowerPoints.

PowerPoint on Kyso

{% hint style="info" %} Note that these file types can be opened in full screen for presentation mode. {% endhint %} ## Publishing Pushing PowerPoints works like any other report type on Kyso. You can upload and categorise your PowerPoint files any which way you prefer: {% content-ref url="/pages/-LjzhPN6Oo\_n\_4fh9D3f" %} [Creating Reports In-App](/posting-to-kyso/creating-reports-on-kyso) {% endcontent-ref %} {% content-ref url="/pages/cA7WA2QbPGWMRdols7Gv" %} [Kyso's Command Line Tool](/posting-to-kyso/kysos-command-line-tool) {% endcontent-ref %} {% content-ref url="/pages/JDCMDT073Dko60xXvGTj" %} [Importing Files from S3](/posting-to-kyso/importing-files-from-s3) {% endcontent-ref %} # RTF Documents How to publish your Word Documents to Kyso! ## OnlyOffice Kyso integrates with OnlyOffice, a free software office suite and ecosystem of collaborative applications that allows the platform to read & render various document types, including Word Documents.

Word Docs on Kyso

## Publishing You can upload and categorise your Word documents any which way you prefer: {% content-ref url="/pages/-LjzhPN6Oo\_n\_4fh9D3f" %} [Creating Reports In-App](/posting-to-kyso/creating-reports-on-kyso) {% endcontent-ref %} {% content-ref url="/pages/cA7WA2QbPGWMRdols7Gv" %} [Kyso's Command Line Tool](/posting-to-kyso/kysos-command-line-tool) {% endcontent-ref %} {% content-ref url="/pages/JDCMDT073Dko60xXvGTj" %} [Importing Files from S3](/posting-to-kyso/importing-files-from-s3) {% endcontent-ref %} **Note that Kyso renders all RTF document types. Microsoft Word is just one example!!!** # Embedded Dashboards Tableau, Google BigQuery dashboards, and more, all indexed for search & discovery on Kyso with version-specific commenting. ## Creating an embedded dashboard On your Organization or Channel homepage, navigate to "Post a report" > "Create embedded report."

Embedding a report

On the next page, supply the URL of where the dashboard is hosted (without the https\://). Select which channel you're publishing to click "Post." This will embed the dashboard as an iframe into a Kyso report. The report's contents are indexed for search, as with any other report type on Kyso. Note that you can also open these embedded dashboards in full screen. {% hint style="info" %} For self-hosted deployments or any dashboard that is not deployed to the same network where your company's Kyso instance is installed, you will need to set up a custom workflow for this to work. Contact the team on the gitlab repository for more info! {% endhint %} # Google Sheets Embedding spreadsheets on Kyso {% hint style="warning" %} Interactivity in Excel files directly uploaded currently does not work on Kyso. For example, users can't play with cell dropdowns or view any other data validation. For that reason, we (for now) recommend to instead simply embed Google Sheets as iframes into Kyso reports until we update the .xlsx renderer. {% endhint %} Note that you need to have access to the original URL (or it's public) to view the spreadsheet in the Kyso iframe.

Google Sheets on Kyso

Note that this will work for any Google document type, for example:

Google Documents

# Looker Dashboards Embedding Looker dashboards on Kyso Looker dashboards will render if you use the embed link. If you try to embed a link from LookerStudio directly into Kyso it will not render. You need to look for the embed link for embedding as an iframe & copy this link into the embed page on Kyso.

Looker Dashboards on Kyso

# Tableau Dashboards Embedding Tableau dashboards on Kyso Rendering Tableau dashboards unfortunately won't work natively since it has its own embed fields that need to be configured, the values of which will be different for each team or organisation. {% hint style="success" %} This is something the Kyso team can do on request, reach out to us on the Gitlab repo! {% endhint %} # Microsoft BI Dashboards Embedding Microsoft BI dashboards on Kyso {% hint style="info" %} Microsoft BI dashboards will only work if you have the explicit link to the app page itself (i.e not to another view of the dashboard). {% endhint %}

Microsoft BI on Kyso

# Plotly's Dash Dash applications embedded into Kyso Embedding Dash applications should work without any additional configurations if you have a link to the app.

Plotly's Dash on Kyso

# Streamlit Embedding Streamlit applications on Kyso Embedding Streamlit apps will work only if you use the embed link explicity (which is not always provided). You need to add `/?embed=true` to the URL for it to work:

Streamlit Applications on Kyso

# Videos How to render video files on Kyso reports {% hint style="info" %} Users can upload mp4s and they’ll render in the report within a markdown file! {% endhint %} ## Instructions 1. On the app, go to **New Report** > **Upload Files**. 2. Upload your e.g. mp4 file. 3. Publish the report. 4. On the report, navigate to the video file in the file browser (if not the main file). The video file has rendered within a markdown file on Kyso. # Creating Reports In-App Create content in the browser just like you would on Sharepoint, Notion or Google Docs There are 2 ways to create knowledge in the browser: ## 1. Upload Existing Files & Folders You can publish existing content by creating new files & folders on the UI, and upload from your local machine. These can be any format that Kyso currently works with: * Jupyter Notebooks * Markdown * HTML * Images * PDFs * PowerPoints * Video files (will render embedded in a markdown document) Simply navigate to "**Post a report"** at the top of your Organization/Channel dashboard and select **"Upload report files"**. {% hint style="info" %} For file types that are relevant to your research post that we might not support **yet**, they will still be hosted and available to download by other users when uploaded. {% endhint %} ## 2. Write Reports from Scratch This is where you can create reports from scratch in Kyso's markdown editor just as you would on Notion, Google Docs or any other general knowledge wiki. Kyso has it's very own built-in markdown editor. On the Organisation or Channel dashboard, simply navigate to "**Post a report"** at the top and select **"Create new report in the UI".** This will take you to `https://kyso.io//create-report/`.

Create a Report

Here you can write up a report from scratch, embed images and code snippets, create new files & folders on the UI, and upload from your local machine. You can even build cool engineering process diagrams like [Mermaid](https://mermaid.js.org/) in the editor! ### Images Simply click on the image icon at the top when you want to include an image. You simply need to supply the URL of the image you're linking to. If you upload an image to the report file browser and want to embed that image in the document, the notation is `./image_name.png`, where `.` is the directory of the main markdown file you are editing. We don't support image links with spaces in the name. So if you link an image in the report, do it like this: ``` ![alt text](image_name.png) ``` and don't do it like this: ``` ![alt text](image name.png) ``` ### Limitations {% hint style="info" %} Note that there is a 5MB limit on the **combined** file size of the report created in the Edit page. This is because these files are stored in local storage during the editing process. If your project is much larger, it is advised to document everything externally & to instead use the upload page to create this report on Kyso, which has a much larger size limit. {% endhint %} # Kyso's Command Line Tool Users can leverage the Kyso command line tool to publish/download reports to and from Kyso directly from their machines. Users can leverage the Kyso command line tool to publish/download reports to and from Kyso directly from the terminal within your data science workspace of choice. This method is particularly useful for those that want to publish results from scheduled processes or connect these reports back to the respective data source. And the idea here is that the Kyso CLI tool can integrate into these workflows. {% content-ref url="/pages/xoc3qpmzkb1Sq89RqjrL" %} [Installation](/posting-to-kyso/kysos-command-line-tool/installation) {% endcontent-ref %} {% content-ref url="/pages/3vJW7o9tpVjInel8DEsg" %} [Advanced configuration](/posting-to-kyso/kysos-command-line-tool/advanced-configuration) {% endcontent-ref %} {% content-ref url="/pages/Ynppwj3gN2Ao1fLO7mIo" %} [Authorization](/posting-to-kyso/kysos-command-line-tool/authorization) {% endcontent-ref %} {% content-ref url="/pages/LEia9hOEUZaGfGYFxZbc" %} [Publishing & Downloading](/posting-to-kyso/kysos-command-line-tool/publishing-and-downloading) {% endcontent-ref %} # Installation How to install the kyso-cli on your machine There are several ways to use **Kyso CLI** in your machine, depending on your operating system, your needs and your preferences. ## NPM & YARN The easiest way to use kyso cli is throught NPM or YARN The **Kyso CLI** can be installed as a NPM or YARN global package easily, by just launching one of the commands below: ``` npm install -g kyso or yarn global add kyso ``` {% hint style="info" %} Depending on your local configuration, you might need to use **sudo or launch the windows terminal with administrative rights.** If that's the case, it's preferrable to review your configuration, but if you are in a hurry you can use **sudo** 😉 {% endhint %} Once installed, check that it's available executing this command: ``` $ kyso Kyso Client VERSION kyso/1.6.0 linux-x64 node-v18.6.0 USAGE $ kyso [COMMAND] TOPICS plugins List installed plugins. COMMANDS clone Clone a report from Kyso help Display help for kyso. import-repository Import repository to Kyso init Interactivel build a kyso.yaml file kyso-command login Login into Kyso open Open a report in the browser plugins List installed plugins. pull Pull repository from Kyso push Upload local repository to Kyso whoami Current logged user and platform ``` ## NPX Since npm version 5.2.0 you can use npx instead of an npm global install. The difference between npm and npx is that npx doesn't install anything globally on your computer, but just downloads the dependency, executes it and then deletes it, keeping your local node\_modules smaller. To use NPX just execute this command: ``` $ npx kyso Need to install the following packages: kyso@1.6.0 Ok to proceed? (y) y Kyso Client VERSION kyso/1.6.0 linux-x64 node-v18.6.0 USAGE $ kyso [COMMAND] TOPICS plugins List installed plugins. COMMANDS clone Clone a report from Kyso help Display help for kyso. import-repository Import repository to Kyso init Interactivel build a kyso.yaml file kyso-command login Login into Kyso open Open a report in the browser plugins List installed plugins. pull Pull repository from Kyso push Upload local repository to Kyso whoami Current logged user and platform ``` ## Windows Installers Visit [https://cli.kyso.io](https://cli.kyso.io/) to download the installers. Download the installer that fits with your architecture and execute it. Keep all the options selected for a better experience! ![Installer components selection](/files/GiJpVfasNhMC67bMnFhz) Select the destination folder and select 'Install': ![Installer location selection](/files/a2GgorWOC4UmMYjuFHaV) ![Installer installing ;)](/files/JiHFE6K6INERwNnMpLAK) Once the installation is completed, select the 'Close' button, open your terminal and write the following command: ``` kyso ``` ![kyso command running](/files/A9GbYv8nla8p7C5QAtvN) ## Tarballs Visit [https://cli.kyso.io](https://cli.kyso.io/) to download the installers. Download the tarball that fits with your operating system and your architecture, extract it in your preferred destination and add it to the PATH of your system. Add the destination folder, plus bin, into the PATH of your system. For example, if your destination folder is `C:\Users\your_user\kyso`*, then* add `C:\Users\your_user\kyso\bin` to the PATH of your system ### Adding kyso to the system path in Windows Type into the search bar **edit environment variables** and choose the suggested result. That should open the following window: ![System settings window](/files/v19gY8M36oGHKP3kDMMP) Click on **Environment Variables** ![Environment variables window](/files/CtjeYTLkab27D8I9LSLJ) Select the **Path** variable of your user, and click **Edit.** ![Edit path environment variable](/files/KklmUsccRSIAw1DtIVwh) Click on **New** and type the full path in which kyso was unzipped, pointing to the **bin** directory. For example, if kyso is located at `C:\Users\Usuario\Downloads\kyso-v1.3.1-9b24ecb-win32-x64`, then the value you should put is `C:\Users\Usuario\Downloads\kyso-v1.3.1-9b24ecb-win32-x64`**`\bin`** ![Added path](/files/YkZ2cPuogc2q0ESn3VIk) Open a terminal and type the command **kyso to check that the installation was completed succesfully:** ![](/files/Wj3czqQd26ofCIPANmVD) ### Adding kyso to the system path in MacOS **and Linux** #### Using zsh terminal Open a ZSH terminal and edit the file `~/.zshrc` to edit the `PATH` variable. Type the full path in which `kyso` was unzipped, pointing to the `bin` directory. For example, if `kyso` is located at `/home/user/kyso-v1.3.1-9b24ecb-linux-x64`, then the value you should put is `/home/user/kyso-v1.3.1-9b24ecb-linux-x64/bin` ``` export PATH="/home/user/kyso-v1.3.1-9b24ecb-linux-x64/bin:$PATH" ``` Check that it's effectively added by executing: ``` cat ~/.zshrc export PATH="/home/user/kyso-v1.3.1-9b24ecb-linux-x64/bin:$PATH" ``` Open a terminal and type the command `kyso` to check that the installation was completed succesfully #### Using bash terminal Open a BASH terminal and edit the file `~/.bashrc` to edit the `PATH` variable. Type the full path in which `kyso` was unzipped, pointing to the `bin` directory. For example, if `kyso` is located at `/home/user/kyso-v1.3.1-9b24ecb-linux-x64`, then the value you should put is `/home/user/kyso-v1.3.1-9b24ecb-linux-x64/bin` ``` export PATH="/home/user/kyso-v1.3.1-9b24ecb-linux-x64/bin:$PATH" ``` Check that it's effectively added by executing: ``` cat ~/.bashrc export PATH="/home/user/kyso-v1.3.1-9b24ecb-linux-x64/bin:$PATH" ``` Open a terminal and type the command `kyso` to check that the installation was completed succesfully ## **Docker** If you don't want to install the client you can also run it using a docker container, the image is available as `kyso/kyso`, to download the latest version of the image execute: ``` $ docker pull kyso/kyso ``` While the container was created mainly to be used in CI/CD systems the following script can be used on **Linux** or **MacOS** to run the `kyso` client as if it were installed on the HOST machine (see the [limitations](#limitations) below): ```shell #!/bin/sh set -e docker run --rm -ti -v "$HOME:$HOME" -e "HOME=$HOME" -w "$(pwd)" \ -u "$(id -u):$(id -g)" --name kyso kyso/kyso kyso "$@" ``` To run the client do the following: ``` $ ./kyso.sh Kyso Client VERSION kyso/1.8.0 linux-x64 node-v16.16.0 USAGE $ kyso [COMMAND] TOPICS plugins List installed plugins. COMMANDS clone Clone a report from Kyso format Format your current report files to add new great features help Display help for kyso. import-repository Import repository to Kyso init Interactivel build a kyso.yaml file kyso-command login Login into Kyso open Open a report in the browser plugins List installed plugins. pull Pull repository from Kyso push Upload local repository to Kyso whoami Current logged user and platform ``` ### Limitations 1. The client executed with the previous script works fine as long as the working directory is under the user's `HOME` directory when it is executed; if that is not the case the directory is going to be the internal version of the `PATH` inside the container, not the host. 2. If the `COMMAND` called needs to launch programs from the `HOST` system it will fail, as we are running inside a container. Currently the commands that fail are `open` and the `login` command when used with a provider that uses the browser (like **Google** or **GitHub**). # Using Amazon Linux Installation steps for Amazon Linux Amazon Linux is a special distribution widely used in Amazon EC2 instances. As we can see in [their documentation](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/setting-up-node-on-ec2-instance.html), the support for NodeJS is not fully compatible with the latest LTS, and this can cause problems with kyso cli, as it is built on top on NodeJS To solve this gap, please check out this [GitHub repository](https://github.com/kyso-io/docker-kyso-in-awslinux) with our tool to install kyso cli in Amazon Linux. # Authorization How to login to Kyso with the Kyso CLI Before executing any action using Kyso CLI, an authentication and authorization process must be fulfilled. Kyso offers different ways to authenticate, and facilitates an interactive mode to allow users to do so easily. To execute the interactive authentication mode just execute `kyso login` and follow the instructions: ``` $ kyso login ? What is the url of your kyso installation? https://kyso.io ? Select a provider (Use arrow keys) ❯ Kyso Access token Google Gitlab ``` {% hint style="danger" %} We do not support logging in with Bitbucket from the CLI due to a limitation of Bitbucket. Bitbucket only accepts one callback URL to verify a login, which is handled by Kyso's API. If you've created a Kyso account on the UI with Bitbucket, you can create an [access token](https://docs.kyso.io/posting-to-kyso/kyso-command-line-tool/authorization#access-token) (see below) and use this token to log in from the command line. {% endhint %} {% hint style="danger" %} We also don't support logging in with Github from the CLI at this time, since Github does not allow us to launch a localhost server to verify login credentials. {% endhint %} First, kyso will ask you for **the URL of your Kyso installation**. Kyso can run on the cloud or on a private installation. So for that reason, before doing any operation, **the Kyso CLI must know where to point**. Once that field is fulfilled, a set of authentication systems can be used. {% hint style="warning" %} Remember to include the **https\://** in this field! {% endhint %} ## Access Token ### Create your token Open your browser, log in with your user and go to your personal settings.

Personal Settings

Click on the "Token" tab on this page.

Access Token

Click on "Generate new Token" button. A new form will appear in your window, asking for the name of your token. Add a descriptive name and click on "Create". A new token will be created and shown at your screen. **Be aware, as this token will not be shown anymore**. Copy it in a secure place and open your terminal. ![New generated token](/files/MRA4jpV1OAtMvoF9CPbB) ### Interactive mode Open your terminal, execute `kyso login` and select **Access token** as the provider. ``` kyso login ? Select a provider (Use arrow keys) Kyso ❯ Access token Google Gitlab ``` You will be asked for your username and your password, and if they are correct, you will be successfully logged into Kyso. ``` kyso login ? What is the url of your kyso installation? https://kyso.io ? Select a provider Access token ? What is your email? your_mail@your_domain.io ? What is your access token (Get one from https://dev.kyso.io/settings )? bfca17ef-ed71-44b8-be95-971ae64aba10 Logged successfully ``` ### Explicit mode Simply run the following command, explicitly stating your deployment URL, login details and access token. ``` kyso login --kysoInstallUrl https://kyso.io --provider kyso --username valid_account@kyso.io --token bfca17ef-ed71-44b8-be95-971ae64aba10 ``` ## Username and Password ### Interactive mode Open your terminal, execute `kyso login` and select **Kyso** as the provider. ``` kyso login ? Select a provider (Use arrow keys) ❯ Kyso Access token Google Gitlab ``` ### Interactive Mode You will be asked for your username and your password, and if they are correct, you will be successfully logged into Kyso. ``` kyso login ? Select a provider Kyso ? What is your username? valid_account@kyso.io ? What is your password? ******* Logged successfully ``` ### Explicit mode Open your terminal and execute the next command: ``` kyso login --kysoInstallUrl https://kyso.io --provider kyso --username valid_account@kyso.io --password your_password ``` {% hint style="danger" %} Using your password in command line tools is not the most secure practice, especially if you are using it in a CI/CD environment. **We strongly recommend using the Access Token alternative instead!** {% endhint %} ## Google #### Only available in interactive mode, & does not work when using the client with docker! Open your terminal, execute `kyso login` and select **Google** as your provider. ``` kyso login ? Select a provider (Use arrow keys) Kyso Access token ❯ Google Gitlab ``` ## Gitlab #### Only available in interactive mode, does not work when using the client with docker! Open your terminal, execute `kyso login` and select **Gitlab** as your provider. ``` kyso login ? Select a provider (Use arrow keys) Kyso Access token Google ❯ Gitlab ``` # Publishing & Downloading How to push & pull your research to and from Kyso using the Kyso CLI The Kyso CLI pushes content to Kyso using a special file named **kyso.yaml** or **kyso.json**. Content in Kyso is structured across: 1. Organizations 2. Channels Channels reside with Organizations. An organization can have multiple channels, and every channel can have an unlimited number of reports and discussions. Kyso currently renders the following document-types (this list will continue to grow): * Jupyter Notebooks * HTML * Markdown * PDFs * PowerPoints * RTF Documents * [Embedded Dashboards](/kysos-renderer/embedded-dashboards) * SVS Microscopic Images ## Publishing Reports Let us illustrate how to push content to Kyso using an example. In this example we have a Jupyter Notebook report, which is in a folder named jupyter-genomics with the following structure: ``` acme-metrics/ ├─ docs/ ├─ images/ ├─ notebooks/ │ ├─ product/ │ │ ├─ main.ipynb ``` To be able to push content, we must create a kyso.yaml file in the root folder **(or** [**in the frontmatter of the notebook in question**](/posting-to-kyso/publishing-faq/pushing-single-files)**)**, in this case, in the jupyter-genomics folder. {% hint style="info" %} Refer to our documentation on **Report Metadata to learn more:** {% endhint %} {% content-ref url="/pages/uIhIHKe0Dl1bwiUeu5UQ" %} [Configuring Report Metadata](/posting-to-kyso/configuring-report-metadata) {% endcontent-ref %} Below is an example of a kyso.yaml file: ``` main: notebooks/product/main.ipynb title: "Plotting Account Activity Levels" description: "Measuring the relationship between team size and various engagement metrics like viewspostsand other actions. Should we be focusing more time on smaller accounts or only on the big fish?" preview: images/preview.png organization: acme channel: general tags: [engagement, revenue] ``` So, in this example, a report named "Plotting Account Activity Levels" will be created within the organization "acme" in the channel "general". Once the kyso.yaml file is ready, you can run the following command to push the content: ``` acme-metrics % kyso push Uploading report '.' Report has 14 files Successfully uploaded report!! ``` Now we can check that the report is successfully uploaded to Kyso:

Published report using the Kyso CLI

## Downloading Reports Conversely, we can also download content from Kyso to our data science workspace from the terminal. Here we have two options: 1. A kyso.yaml/json is in our current directory (with the relevant metadata) such that, when we run `kyso pull`. Kyso will read the metadata to know which report to download. 2. You wish to update an existing report on Kyso. You can run the `kyso clone` command and explicitly state the project report you're looking for: ``` kyso clone Downloading the report... Successfully downloaded the report!! ``` {% hint style="info" %} Note that on a Kyso report, you can click **"Clone"** and copy the command to your clipboard. {% endhint %}

Kyso Clone

Once you've made your changes, you can push the report back up to Kyso, which will render your latest version by default. # Advanced configuration This page explains how you can configure your environment to start publishing research to Kyso using our CLI package. Kyso runs on premise with its own domain, hosted on your company's own servers. For that reason, the **Kyso CLI** can point to multiple instances of Kyso, and before starting to use the **Kyso CLI** you need to define to which instance you want to point. ## Linux and MacOS ### Using zsh terminal Open a ZSH terminal and edit the file **\~/.zshrc** to add the following environment variable: ``` export KYSO_API=https://kyso.io/api/v1 ``` Check that it's effectively added by executing: ``` cat ~/.zshrc export PATH="/usr/local/opt/openjdk/bin:$PATH" export KYSO_API=https://kyso.io/api/v1 ``` Close your terminal and open it again (as your current instance of the terminal is not updated until you restart it), and then execute the following command to check that the result is the same: ``` echo $KYSO_API https://kyso.io/api/v1 ``` Now all the operations of **Kyso CLI** will point to the defined instance. {% hint style="info" %} Remember that you can change the value of KYSO\_API to your company's on premise instance of Kyso! {% endhint %} ## Using bash terminal Open a BASH terminal and edit the file **\~/.bashrc** to add the NEXT environment variable: ``` export KYSO_API=https://kyso.io/api/v1 ``` Check that it's effectively added by executing: ``` cat ~/.bashrc export PATH="/usr/local/opt/openjdk/bin:$PATH" export KYSO_API=https://kyso.io/api/v1 ``` Close your terminal and open it again (as your current instance of the terminal is not updated until you restart it), and then execute the following command to check that the result is the same: ``` echo $KYSO_API https://kyso.io/api/v1 ``` Now all the operations of **Kyso CLI** will point to the defined instance. {% hint style="info" %} Remember that you can change the value of KYSO\_API to your company's on premise instance of Kyso! {% endhint %} ## Windows In order to use and set environment variables in Windows, follow the next steps: * Select Windows+R to open Run dialog * Type **sysdm.cpl** and press OK to open a System Properties dialog ![Windows System Properties](/files/jvGwwAKQbIc6tzIMe0oz) * Go to **Advanced > Environment Variables** ![Setting Windows Environment Variables](/files/W5nuj0ZTkhBDImjlfq3d) * Click the "New" button in System variables section, and fill in the text-boxes with the following contents: * Variable name: KYSO\_API * Variable value: {% hint style="info" %} Remember that you can change the value of KYSO\_API to your company's on premise instance of Kyso! {% endhint %} # Integrating with Git This section will help you understand Kyso's Git integration so you can start automatically publishing reports on commit. ## **Kyso integrates with all 3 Git systems:** {% content-ref url="/pages/tRU6GKqaWClzv4Wq89uz" %} [Github](/posting-to-kyso/connect-a-github-repo-to-kyso/github) {% endcontent-ref %} {% content-ref url="/pages/CqbJFVI2QggqqyFnCdpb" %} [Gitlab](/posting-to-kyso/connect-a-github-repo-to-kyso/gitlab) {% endcontent-ref %} {% content-ref url="/pages/ddRMWqxeSDHRZhrBFPGP" %} [Bitbucket](/posting-to-kyso/connect-a-github-repo-to-kyso/bitbucket) {% endcontent-ref %} ## How does it work? Kyso uses Git actions & CI pipelines with each VCS. This is a super-powered way of integrating into your existing workflows, allowing you to fine tune how you commit your work to Kyso - you can choose to integrate a specific branch, tag, on an event - or any other specification of your choosing. You can import both public and private repositories from Git. ## Git metadata capture When you create a report on Kyso, in the browser or from the Command Line, report metadata is created automatically. This is so users can browse through previous iterations of a project and track where & when changes are made, and by whom. For Git–based reports, we've updated the report interface to also display the commit hash and provide links back to the source Git repository. No additional action nor configuration is needed from the user. Simply integrate the Kyso CLI into your Git CI pipelines ([see our docs on how to do that here](/posting-to-kyso/connect-a-github-repo-to-kyso/github)), and Kyso will handle the rest. This is how a Git-based report now looks on Kyso:

Git Links

Report Version & Commit Hash

In the image on the left you can see both the link back to the main Git repository and to the specific commit hash of that report version. On the right, users can browse through previous iterations of the project on Kyso, with each version displaying it's own Git metadata, such as commit hash & creation date. ## Kyso Metadata Configuration Remember that in order to ensure a successful import process, your repositories will need to be set up so Kyso can read in the metadata and properly structure your content. This is especially important if you're connecting multiple repositories or one repository with a lot of sub-directories intended to be published as their own individual Kyso reports. Read more about report metadata here: {% content-ref url="/pages/uIhIHKe0Dl1bwiUeu5UQ" %} [Configuring Report Metadata](/posting-to-kyso/configuring-report-metadata) {% endcontent-ref %} # Github Learn how to set up Github Actions so publishing your data science work becomes automatic!! ## Prerequisites * Have a Kyso account - either on kyso.io or on your company's private Kyso installation. * Create a Kyso access token - follow these [instructions](https://docs.kyso.io/posting-to-kyso/kyso-command-line-tool/authorization#access-token). **Save this for later!** * Ensure your directory (or [folders](/posting-to-kyso/publishing-faq/meta-reports)) contains a valid `kyso.yaml` file. Check the following [instructions](https://docs.kyso.io/posting-to-kyso/configuring-report-metadata) for more info. * You can instead include your Kyso metadata in the frontmatter of a notebook or markdown file for each project report. Kyso will search for this information on a push action. ## How to publish to Kyso with Git Actions {% hint style="info" %} About the `kyso-push` step variables: * The`username`and `token` fields take their value from secret variables, to make the system work the user has to create the kyso auth token and define the `KYSO_USERNAME` and `KYSO_TOKEN` as explained on the [github documentation](https://docs.github.com/en/actions/security-guides/encrypted-secrets). * The`url` has to point to your **Kyso** deployment. So if your company, Acme Inc. has its own **Kyso** instance available on , that is the value that has to be assigned to it. {% endhint %} 1. Create a `.github/workflows` directory in your repository on GitHub if this directory does not already exist. 2. In the `.github/workflows` directory, create a file named `kyso-action.yml`. 3. Copy the following YAML contents into the `kyso-action.yml` file: ```yaml name: Kyso Push on: push: branches: - main jobs: Kyso-Push: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - uses: actions/setup-node@v3 with: node-version: 18 - run: echo ${{secrets.KYSO_TOKEN}} - run: npm install -g kyso - run: kyso login --kysoInstallUrl https://kyso.io --provider kyso --username kyle@kyso.io --token ${{secrets.KYSO_TOKEN}} - run: kyso push ``` {% hint style="warning" %} Remember to create a repository environment variable as shown above to login in to Kyso from the CLI, rather than explicitly stating your access token in the actions script, which would potentially breach best security practices. {% endhint %} This means that on each new **push** to the repository, a Github action will be generated to run the commands in our action file above. Remember to also ensure that your metadata specs in the sub-directory YAML files (or notebook frontmatter), are correct: * organization: \[Your Organisation Name] * team/channel: \[Destination channel for each report, and this might be different for each sub-directory] * author: \[User email, or a list or emails for multiple authors] {% hint style="warning" %} Note that the example repo contains multiple sub-folders and so take on the `type: meta`, and each sub-folder will have it's own kyso.yaml file with specifications on that individual project's metadata (e.g. title, description, type, tags, etc.). Read more at the link below: {% endhint %} {% content-ref url="/pages/SWRKSQwfvfB1pUCYczFh" %} [Meta Reports](/posting-to-kyso/publishing-faq/meta-reports) {% endcontent-ref %} ## Executing the CI/CD Pipeline When we commit our work to Github, an Action will queued to execute. Navigate to the actions page on the repo (`https://github.com/user/repo/actions`), we'll see that our runs are executing, first by installing the Kyso CLI, logging the user in, and then publishing the report. If we check the logs, this is what we will see: ``` $ npm install -g kyso added 234 packages, and audited 235 packages in 19s 33 packages are looking for funding run `npm fund` for details found 0 vulnerabilities $ kyso login --kysoInstallUrl https://kyso.io --provider kyso --username [YOUR USERNAME] --token [YOUR TOKEN] Logged successfully $ kyso push 9 reports found No new or modified files to upload in report 'Salesforce Pipeline Looker Dashboard' No new or modified files to upload in report '10xgenomics HTML Report: Chromium Nuclei Analysis' Uploading report 'jupyter-notebooks' 🎉🎉🎉 Report Jupyter Notebook: Graphing Mutation Ratios was uploaded to: https://kyso.io/kyso-demo/data-analyses/jupyter-notebook-graphing-mutation-ratios 🎉🎉🎉 No new or modified files to upload in report 'HTML Outputs: MultiQC Sequencing Data' No new or modified files to upload in report 'PDF Report: Hurdle & Zero Inflated Models, Overdraft Analytics' No new or modified files to upload in report 'PowerPoint Presentation: Knowledge Aggregation with Kyso' Uploading report 'markdown-writeups' 🎉🎉🎉 Report Markdown Report: Engineering Process Diagrams was uploaded to: https://kyso.io/kyso-demo/data-analyses/markdown-report-engineering-process-diagrams 🎉🎉🎉 Uploading report 'nba-player-clustering' 🎉🎉🎉 Report NBA Player Clustering was uploaded to: https://kyso.io/kyso-demo/data-analyses/nba-player-clustering 🎉🎉🎉 Uploading report 'baseball-data-analysis' 🎉🎉🎉 Report Baseball Data Analysis was uploaded to: https://kyso.io/kyso-demo/data-analyses/baseball-data-analysis 🎉🎉🎉 ``` Now we can navigate to Kyso at to see the published list of our example reports! ## Maintaining a QA process with Pull Requests By integrating Kyso into Git actions, the workflow is super flexible and you can refine this further and select if you want to integrate a specific branch, tag, or whatever else fits your specific workflow needs! Here is a link to all the different event types that trigger these workflows: {% embed url="" %} ## Publishing only when a PR is merged into the main branch So, for example, if you work with a team of scientists, data scientists, etc.., all pushing and pulling to and from the same repository, you're going to want to control how, when and what changes get published to Kyso. Now there is ongoing debate on how to do this. For example, see this discussion: {% embed url="" %} Github > Kyso on Merge Workflow {% endembed %} There are a couple of different commands we can use to ensure that only when a PR is merged into our main branch is our workflow triggered. However, we can actually just use our existing action file: ``` name: Kyso Push on: push: branches: - main jobs: Kyso-Push: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - uses: actions/setup-node@v3 with: node-version: 18 - run: npm install -g kyso - run: kyso login --kysoInstallUrl https://kyso.io --provider kyso --username [YOUR USERNAME] --token [YOUR ACCESS TOKEN] - run: kyso push ``` Because a merged pull request always results in a push, we can just use the push event to accomplish our goal. So the above workflow will run when a PR is merged or when a commit is made directly to the master branch. To make this workflow even more secure, it has been recommended that you **add branch protection rules to your main branch:** {% embed url="" %} Branch Protection Rules {% endembed %} # Gitlab Learn how to setup Gitlab CI pipelines so publishing your data science work becomes automatic!! ## Prerequisites * Have a Kyso account - either on kyso.io or on your company's private Kyso installation. * Create a Kyso access token - follow these [instructions](https://docs.kyso.io/posting-to-kyso/kyso-command-line-tool/authorization#access-token). **Save this for later!** * Ensure your directory (or [folders](/posting-to-kyso/publishing-faq/meta-reports)) contains a valid `kyso.yaml` file. Check the following [instructions](https://docs.kyso.io/posting-to-kyso/configuring-report-metadata) for more info. ## How to publish to Kyso with Gitlab CI Pipelines {% hint style="danger" %} About the `kyso-push` step variables: * The`username`and `token` fields take their value from secret variables, to make the system work the user has to create the kyso auth token and define the `KYSO_USERNAME` and `KYSO_TOKEN` as explained on the [github documentation](https://docs.github.com/en/actions/security-guides/encrypted-secrets). * Create your Access Token according to this document: * The`url` has to point to your **Kyso** deployment. So if your company, Acme Inc. has its own **Kyso** instance available on , that is the value that has to be assigned to it. {% endhint %} 1. Create a `.gitlab-ci.yml` file in your Gitlab repository. 2. Copy the following YAML contents into `.gitlab-ci.yml`: ``` image: node:18 Upload to Kyso: script: - npm install -g kyso - kyso login --kysoInstallUrl https://kyso.io --provider kyso --username [YOUR EMAIL] --token [YOUR ACCESS TOKEN] - kyso push ``` {% hint style="danger" %} We're using node version 18 here, note that this might need to change if looking at this doc in the future. This works as of September 27th 2022. {% endhint %} This means that on each new **push** to a tag on the repository, a CI pipeline will be generated to run the commands in our action file above. Remember to also ensure that your metadata specs in the sub-directory YAML files, are correct: * organization: \[Your Organisation Name] * team/channel: \[Destination channel for each report, and this might be different for each sub-directory] {% hint style="info" %} Note that if your repository contains multiple sub-folders, it will be of t`ype: meta` in the root directory `kyso.yaml` file and each sub-folder will have it's own kyso.yaml file with specifications on that individual project's metadata (e.g. title, description, tags, etc.). Read more here: {% endhint %} {% content-ref url="/pages/SWRKSQwfvfB1pUCYczFh" %} [Meta Reports](/posting-to-kyso/publishing-faq/meta-reports) {% endcontent-ref %} ## Executing the CI Pipeline When we commit changes to the Gitlab repository, a Docker image is created to run our job. We can check out the pipeline logs at `https://gitlab.com/user/repository/-/jobs/job-id` and should see something like this: ```bash Running with gitlab-runner 15.4.0 (43b2dc3d) on Gitlab Shared Runner (Docker) Q2W747Cx Preparing the "docker" executor 00:03 Using Docker executor with image node:18 ... Pulling docker image node:18 ... Using docker image sha256:5eb5251ba1b16c9549a23229d4fb1211e2a7f4ca6ce7b91fb86d939fd886440e for node:18 with digest node@sha256:d6ed353d022f6313aa7c3f3df69f3a216f1c9f8c3374502eb5e6c45088ce68e8 ... Preparing environment 00:00 Running on runner-q2w747cx-project-94-concurrent-0 via 276b460ab7aa... Getting source from Git repository 00:01 Fetching changes with git depth set to 20... Reinitialized existing Git repository in /builds/kyle/kyso-demo/.git/ Checking out 433188c3 as main... Skipping Git submodules setup Executing "step_script" stage of the job script 00:20 Using docker image sha256:5eb5251ba1b16c9549a23229d4fb1211e2a7f4ca6ce7b91fb86d939fd886440e for node:18 with digest node@sha256:d6ed353d022f6313aa7c3f3df69f3a216f1c9f8c3374502eb5e6c45088ce68e8 ... $ npm install -g kyso added 236 packages, and audited 237 packages in 13s 33 packages are looking for funding run `npm fund` for details found 0 vulnerabilities $ kyso login --kysoInstallUrl https://kyso.io --provider kyso --username kyle@kyso.io --token my-kyso-token Logged successfully $ kyso push 4 reports found No new or modified files to upload in the 'rna-sequences' folder. 🎉🎉🎉 Report was uploaded to https://kyso.io/kyso-examples/life-sciences/graphing-genomic-mutation-ratios 🎉🎉🎉 No new or modified files to upload in the 'small-molecules' folder. No new or modified files to upload in the '10xgenomics' folder. Cleaning up project directory and file based variables 00:00 Job succeeded ``` ## Maintaining a QA process with Pull Requests Similar to our documentation on Github Actions, [we can maintain a QA process, ensuring only merged/reviewed work is pushed upstream to Kyso.](https://docs.kyso.io/posting-to-kyso/connect-a-github-repo-to-kyso/github#maintaining-a-qa-process-with-pull-requests) {% content-ref url="/pages/tRU6GKqaWClzv4Wq89uz" %} [Github](/posting-to-kyso/connect-a-github-repo-to-kyso/github) {% endcontent-ref %} So, for example, if you work with a team of scientists, data scientists, etc.., all pushing and pulling to and from the same repository, you're going to want to control how, when and what changes get published to Kyso. {% hint style="success" %} Because a merged pull request always results in a push, we can just use the push event to accomplish our goal. So the above workflow will run when a PR is merged or when a commit is made directly to the master branch. {% endhint %} To make this workflow even more secure, it has been recommended that you **add branch protection rules to your main branch!** # Bitbucket Learn how to setup CI pipelines so publishing your data science work becomes automatic!! ## Prerequisites * Have a Kyso account - either on kyso.io or on your company's private Kyso installation. * Create a Kyso access token - follow these [instructions](https://docs.kyso.io/posting-to-kyso/kyso-command-line-tool/authorization#access-token). **Save this for later!** * Ensure your directory (or [folders](/posting-to-kyso/publishing-faq/meta-reports)) contains a valid `kyso.yaml` file. Check the following [instructions](https://docs.kyso.io/posting-to-kyso/configuring-report-metadata) for more info. ## How to publish to Kyso with Bitbucket CI Pipelines {% hint style="danger" %} About the `kyso-push` step variables: * The`username`and `token` fields take their value from secret variables, to make the system work the user has to create the kyso auth token and define the `KYSO_USERNAME` and `KYSO_TOKEN` as explained on the [github documentation](https://docs.github.com/en/actions/security-guides/encrypted-secrets). * Create your Access Token according to this document: * The`url` has to point to your **Kyso** deployment. So if your company, Acme Inc. has its own **Kyso** instance available on , that is the value that has to be assigned to it. {% endhint %} In the root directory of the repository, create a`bitbucket-pipelines.yml` that details the actions we want to run and when: ``` image: node:18 pipelines: branches: main: - step: name: 'Publishing to Kyso' trigger: 'automatic' script: - npm install -g kyso - kyso login --kysoInstallUrl https://kyso.io --provider kyso --username [YOUR EMAIL] --token [YOUR ACCESS TOKEN] - kyso push ``` {% hint style="warning" %} Note that \`--kysoInstallUrl\` needs to point to your company's Kyso deployment. So if your company, Acme Inc., has their Kyso on , then that is your install url address! {% endhint %} So now every time we make a commit to this repository, whether on the UI or from the Command Line, the above script will be run, publishing and updating our work in Kyso. Note that this is a super simple template using the example Starter pipeline configuration, but this can be integrated into much more powerful pipeline flows. For example, you can decide to only push specific branches or tags. {% hint style="info" %} Don't see or can't figure out how to set up your desired workflow? Simply reach out to us, let us know your use case and how you want your pipeline to work, and we'll send you the documentation!! {% endhint %} ## Executing the CI Pipeline Using our example repository from above, as we commit changes to the Bitbucket project, a Docker image is created to run our job. We can check out the pipeline logs at `https://bitbucket.org/user/repository/pipelines`. Clicking into one of the most recent successful merges, we should see something like this:

Bitbucket Pipeline

## Maintaining a QA process with Pull Requests Similar to our documentation on Github Actions, [we can maintain a QA process, ensuring only merged/reviewed work is pushed upstream to Kyso.](https://docs.kyso.io/posting-to-kyso/connect-a-github-repo-to-kyso/github#maintaining-a-qa-process-with-pull-requests) {% content-ref url="/pages/tRU6GKqaWClzv4Wq89uz" %} [Github](/posting-to-kyso/connect-a-github-repo-to-kyso/github) {% endcontent-ref %} So, for example, if you work with a team of scientists, data scientists, etc.., all pushing and pulling to and from the same repository, you're going to want to control how, when and what changes get published to Kyso. {% hint style="success" %} Because a merged pull request always results in a push, we can just use the push event to accomplish our goal. So the above workflow will run when a PR is merged or when a commit is made directly to the master branch. {% endhint %} To make this workflow even more secure, it has been recommended that you **add branch protection rules to your main branch!** # Configuring Report Metadata How to ensure your colleagues can easily discover your company's data science work. All reports on Kyso contain variables like authors, dates, tags, descriptions, and titles as metadata. This metadata can be added in various ways: 1. Directly on reports in the Kyso UI (titles, descriptions and tags only). 2. Include it in a kyso.yaml/kyso.json file **in the same directory as the report.** {% hint style="info" %} A third way will be possible in one of our upcoming deployments - in the first cell (JSON) of a Jupyter notebook, its "front matter." {% endhint %} And end-users can search across your company's knowledge base by this report metadata. ## Properties ### TL/DR Here is minimal kyso.yaml file, which includes the most important variables: * `organization` * `channel` (can also be denoted `team`) ``` organization: acme channel: general title: "Project Title" main: report.ipynb ``` Below is an example of a comprehensive YAML file: ``` organization: acme author: kyle@kyso.io channel: general title: "Project Title" description: "Project Description." main: report.ipynb preview: "images/preview.png" tags: [tag1, tag2, tag3] ``` #### **Property Lexicon:** * **main**: *required*. File which will be rendered automatically when the project is opened on Kyso. * **title**: *required*. Name of the report. * **description**: *required*. Description of the report. * **organization**: *required*. Organization which owns the report. * **team**: *required*. Team within the organization to which the report belongs. * **type**: *required if 'meta'*. Type of report. * **meta**: Mono-folder in which each sub-directory is published as separate project report. * **reports**: *required* only if `type: meta` * ``` reports: - folder1 - folder2 ``` * In this case, each sub-folder should have its own yaml/json file for the individual reports. * **preview**: *optional*. Preview image of the report. One will be generated automatically in its absence. * **author**: *optional.* If no author is specified, the logged in user pushing from the command line will be set as the sole author. {% hint style="success" %} You can specify multiple **authors** as a list! authors: \["", ""] {% endhint %} {% content-ref url="/pages/SWRKSQwfvfB1pUCYczFh" %} [Meta Reports](/posting-to-kyso/publishing-faq/meta-reports) {% endcontent-ref %} Note that you are also able configure the title, description, and preview image on the post on Kyso after import, but it is recommended to include this in the YAML header for a more seamless workflow. **Recommendation:** Always add a title, description and preview image to your posts. You will receive a lot more readers. If you want to validate your YAML before pushing to Github - checkout this [YAML Validator](http://www.yamllint.com/). {% hint style="warning" %} Users need to manually add authors to the YAML file when updating reports from a new profile. Kyso does NOT automatically detect a new author on later versions if pushed from another account. {% endhint %} ## Example Workflow I am working on my local machine in a directory Users/Kyle/Data-Analysis. This folder contains multiple sub folders, which all denote separate projects and that I want to publish as their own Kyso reports. Example: ``` Data-Analysis └── Marketing └── data/ └── report.ipynb └── images/ └── Product └── data/ report.ipynb └── images/ etc.. ``` In my root directory (Users/Kyle/Data-Analysis), I have the following kyso.yaml file: ``` type: meta reports: - marketing - product ``` This tells Kyso to import 'Marketing' and 'Product' as their own individual Kyso reports and not 'Data-Analysis' as the one Kyso report. By default Kyso does not import in the root directory (Data Analysis) as its own report, but rather acts as a navigation system for Kyso in finding the project reports. Then in each sub-directory I have another kyso.yaml which sets the metadata for that specific report. For example, in 'Marketing' I could have the following" ``` organization: acme team: marketing description: "This report explores how we can pull in website data from Google Analytics and find insights on SEO." author: kyle@kyso.io main: report.ipynb preview: images/preview.png title: "Predicting the results of our latest marketing campaign" tags: [seo, content-campaigns, google-analytics] ``` And something similar in 'Product'. Any time changes are made to any individual report, Kyso will version the projects as you publish again and again. ## The Power of Tagging As seen above, authors can 'tag' their reports within the metadata. What makes this specific variable so useful is that when users are searching for reports on Kyso, using specific tags when combined with searching by team and full-text (see next two documents) will really narrow a search from '000s down to a few relevant reports. ## File Header Metadata In the absence of any kyso.yaml file in the directory, Kyso will look for a YAML header in a notebook's or markdown file's metadata. This may be a better option for those of you posting ongoing projects, to which commits are made on a daily or weekly basis. Rather than manually configuring a YAML file, you can simply update the metadata while working in the notebook. {% content-ref url="/pages/YsuG5XbwSHcRBl6wdnDT" %} [Pushing Single Files](/posting-to-kyso/publishing-faq/pushing-single-files) {% endcontent-ref %} {% hint style="info" %} Note that this currently only works for Jupyter- and Markdown-based reports. {% endhint %} The example below is our YAML configs contained within 3 dashed (-) lines top and bottom, in **the first Markdown cell** of the notebook or markdown file: ``` --- organization: acme channel: marketing title: "My awesome post" description: "This is a description of what I did in my awesome post" main: "my-article.ipynb" tags: [apples, oranges] --- ``` # Importing Files from S3 Does your company currently have 000s of PowerPoints, Notebooks, PDFs and other file types just sitting in an s3 bucket? {% hint style="danger" %} **Note that this feature is available upon request only.** {% endhint %} Our team has built a mass import option for pulling 00s, if not 000s of existing files currently stored in s3 buckets. See the gif below for a beta view of how it works on the UI: You simply need your AWS key and path to the folder(s) on S3 and can create your own metadata mappings so all files are automatically categorised into Kyso Organizations and Channels on import. You can set default values for any file with missing metadata. And the files are immediately indexed full-text for search. This workflow can work for a 1-time import OR a continuous integration, in which Kyso pulls in edited files in an s3 folder on a periodic basis or when executed. {% hint style="info" %} This workflow can also just be run from the Command Line with the Kyso CLI if preferred to the UI option above. {% endhint %} # Publishing FAQ Troubleshooting publishing issues Having problems pushing your reports to Kyso? See if any of the documents below can guide you! {% content-ref url="/pages/SWRKSQwfvfB1pUCYczFh" %} [Meta Reports](/posting-to-kyso/publishing-faq/meta-reports) {% endcontent-ref %} {% content-ref url="/pages/YsuG5XbwSHcRBl6wdnDT" %} [Pushing Single Files](/posting-to-kyso/publishing-faq/pushing-single-files) {% endcontent-ref %} {% content-ref url="/pages/4mU6qYZ1uhwqc8AlLspa" %} [Issues with Report Rendering](/posting-to-kyso/publishing-faq/issues-with-report-rendering) {% endcontent-ref %} # Meta Reports How to configure metadata on multi-level repositories when publishing to Kyso. Meta reports are a mono-folder in which each sub-directory is published as separate project report. To create a meta report include the following in your kyso.yaml file: ``` type: meta reports: - folder1 - folder2 ``` In this case, each sub-folder should have its own yaml/json file for the individual reports. This is particularly useful for users publishing to Kyso via Git actions, since a lot of companies tend to have one or a few large mono-repos specifically for data science work. ## Multi-level Report Configuration It is quite popular for companies to maintain all there data science research in a single mono-repository (or just a few for much bigger organisations) in which there are multiple sub-directories that pertain to different projects, and within each of these projects are the notebook reports and associated files. It makes sense to want to import these large repositories such that each sub-project is imported to Kyso as it's own project report. And it is quite straightforward to do this!:) There are only 2 additional fields you will need to add to the yaml file in the root directory to configure this behaviour. 1. `type: meta` 2. `reports` **type**: This tells Kyso that there are multiple repositories in the repo **reports**: This will be a list of folders you want to import as separate Kyso reports. Currently the child posts must be organised in folders, importing different files from one folder as different child posts is not possible. ``` report: - folder1 - folder2 ``` You can use wildcards, so for example, if you wanted to include all folders in the root directory just use: ``` report: - "*" ``` Or if you wanted to include all folders starting with "article" use: ``` report: - "article*" ``` ## Default Values If the **meta** kyso.yaml file has other properties like: * channel * organization * tags they will be treated as **default values**, which means if the `report/folder/kyso.yaml` doesn't have a specific value for that property, the default one will be used. If not, the `report/folder/kyso.yaml` properties values will overwrite the defaults. ## Overriding configurations in the child reports You can add a *kyso.yaml* to each child folder and override any of the defaults the same as you do when you are importing just one repo. For example, you could have a repository with *kyso.yaml* in the root directory: ``` main: main.ipynb posts: - "*" ``` Above we've specified the default name for all main files and are pulling in all folders within the repository. In any given folder, we could have another *kyso.yaml* like so: ``` title: "Project Title" description: "Project description." main: notebook.ipynb preview: "images/preview.png" tags: [ tag1, tag2, tag3 ] ``` # Pushing Single Files Sometimes you may want to post a single file for some quick testing instead of an entire folder every time you push... Users can publish single files to Kyso from the Command Line. In your terminal, simply navigate to the folder where your file is located. Let's say I have a project directory with the following structure: ``` folder notebook1.ipynb notebook2.ipynb notebook3.ipynb notebook4.ipynb notebook5.ipynb data.txt presentation.pdf ``` I want to share a graph from one of my notebook files with a colleague for some quick feedback -> without uploading the entire project. To do this I simply need to run: ``` kyso push -p notebook1.ipynb ``` {% hint style="info" %} Note that you'll need to be logged in to Kyso with the CLI tool and **the file will need to contain valid metadata in the header for Kyso to be able to index it accordingly.** {% endhint %} The following might be additional relevant documentation: {% content-ref url="/pages/cA7WA2QbPGWMRdols7Gv" %} [Kyso's Command Line Tool](/posting-to-kyso/kysos-command-line-tool) {% endcontent-ref %} {% content-ref url="/pages/uIhIHKe0Dl1bwiUeu5UQ" %} [Configuring Report Metadata](/posting-to-kyso/configuring-report-metadata) {% endcontent-ref %} # Issues with Report Rendering If your post is not rendering the way you expected the most common issues are related to the specifications in the kyso.yaml file. If your reports are not appearing the way you expect - make sure that the specs in the `kyso.yaml` file are correct. You can verify everything with the [YAML Validator](http://www.yamllint.com/). Review what you need in your yaml file at the following links: {% content-ref url="/pages/uIhIHKe0Dl1bwiUeu5UQ" %} [Configuring Report Metadata](/posting-to-kyso/configuring-report-metadata) {% endcontent-ref %} {% content-ref url="/pages/SWRKSQwfvfB1pUCYczFh" %} [Meta Reports](/posting-to-kyso/publishing-faq/meta-reports) {% endcontent-ref %} # Searching Reports Kyso's search engine is structured in such a way that users can easily find the content they're looking for, even when filtering through '000s of reports! ## Searching in Channels In any channel you have access to, you can simply select which variables you want to filter by with Boolean selectors, similar to how Gitlab's search works. You can refine your search further with text input that will search by title and description content.

Metadata Selectors to Narrow Search

## Global, Full-Text Search Using Elastic Search, Kyso also indexes full-text on the report body, such that end users are able to search by the internal content of project reports, on all reports across all Organisations and Channels one has access to. Simply enter your search query in the top navigation bar input box - Kyso's global search. You can enter keywords or longer-tail strings and Kyso will return matching results.

Kyso's Global Search Page

As you can see from the image, comments are also indexed and searchable. The conversations that insights inspire are just as important to maintain as the insights themselves. Kyso ranks the most relevant results at the top of the list. {% hint style="success" %} Note that your search queries are saved to the url, which you can share as you would a Kyso report!!! {% endhint %} You can narrow full-text search queries even further by selecting specific metadata like channels, tags, authors, and more. You can even filter by file type. This allows the user to construct really complex search queries. # Browsing Files & Versions This page describes the viewing options for readers on Kyso reports ## File Browsing Readers can browse through the different files attached to the report. Note that, by default, all files within a repository (folder) are imported into Kyso. To ensure certain files are not included in the import (example: images, data files, etc.) add a **`.kysoignore`** file to the directory and Kyso will not import those files. You can have as many files, notebooks, etc.. within a project as you like.

Browsing Files, Versions

## Versions For all reports published to Kyso, any new commits made will be reflected here. Kyso shows the latest version of a report by default, such that: * All changes are tracked and maintained. * Users can view previous versions of the report. So this means that *analyses/research is always up to date and the project's lifecycle is maintained.* {% hint style="info" %} The main file on a report can change when a new version is pushed, if that is preferred. Imagine a data science project workflow, whereby a user documents their thoughts, requirements, etc. in Kyso's markdown editor. Later, they can clone the report from Kyso, move to the next stage of the project - e.g. modelling data in a Jupyter notebook. They can push the report back upstream to Kyso, now making the notebook the report's main file on this new version. {% endhint %} # Report Comments How to collaborate with your colleagues on a Kyso report {% hint style="info" %} Note that depending on your individual access level, you might not have the permission to make comments or create tasks! {% endhint %} Users can also collaborate project-wide with colleagues by making comments at the bottom of the reports.

Kyso Report Comments

Think of this similar to a Reddit-style discussion thread. Here you can ask questions, provide feedback to the authors and discuss results. Every time a new comment is made, the author receives an email notification. If someone responds to your individual comment, you will also receive an email notification. You can tag other users on the team too. {% hint style="info" %} Comments made here apply to the entire project report, and will persist regardless of which file is currently open on the report. {% endhint %} # Report Tasks How your team can manage Gitlab-esque tasks in comment threads directly on Kyso reports {% hint style="info" %} Note that depending on your individual access level, you might not have the permission to make comments or create tasks! {% endhint %} Users can make comments on specific files on the right hand side of a report. While global report comments are designed for more general discussion, these inline comments should be reserved for specific actions - or tasks - on a report.

File-specific Commenting

It is important for users to remember that these comments are file specific. So if one makes a comment on the main PowerPoint file, but then browses to a separate .png file in the report, that comment will disappear from view while browsing. {% hint style="info" %} Note that users can also apply comment statuses to file comments (i.e. comment threads on the right-hand side of a report, which pertain to the specific file that is currently open in the report. General report comments (at the bottom of a report) do not have task statuses. {% endhint %}

Task Statuses

## Jupyter Notebooks Jupyter-based reports have inline comments / tasks. Users can comment directly on a specific Jupyter notebook cell on Kyso - whether that be code input or a graphical output - just as they would on Google Docs or Notion.

Inline Comments

Remember that, because inline comments rely on cell ids, if one makes a comment on a specific cell, and later the author makes changes/updates the report, deleting that cell in the process, that comment will also be removed when the new version is pushed back to Kyso. ## Managing Tasks As multiple versions are created on a Kyso report, users need a way to view & interact with a list of all comments on the report across different versions. We have created a report tab where users can view all comments made across a report’s entire lifecycle. Users can also now apply comment "statuses" to file & inline comments (similar to Google Docs), so teams can manage ongoing work needed on a project. At the top of the report page, the author (or admins) can view a "Report Tasks" page to see all comments made on the report:

Tasks tab on a Kyso report

And users can view all tasks here across report versions, either as a list or a kanban view:

Kanban View

List View

When a user clicks on a task, they are taken to that task with a status history of the comment thread & the cell or file to which the task pertains:

Task & comment thread history

## Orphaned Report Tasks Remember that because tasks are linked to a specific cell in a notebook report, if that cell is deleted on a new version of the report, this effects how you interact with that task. Following the same steps from above, let's say you create a task on a report. In a later version, that cell is deleted. If you - or any other user - clicks on that task in the report tasks page, **this will link you to the latest version of the report when the cell to which that task relates still exists.** ## Managing User Tasks Similar to a platform like Gitlab, users can also view, manage and interact with all tasks that have been assigned to them across all Organisations & Channels to which they have access. Simply click on the tasks icon in the top navigation bar:

Tasks Overview

On the next page you will see all tasks that are relevant to you. You can filter through these tasks just like you would on Gitlab, by task author, status, creation date & Organisation.

User Tasks Page

# Notebook Report Snippets Kyso leverages Jupyter's new unique cell ids so users can share specific sections of a report with their colleagues. By leveraging a notebook's cell identifiers, users can share **Report Snippets** directly. This is especially useful when sharing notebook reports that contain lots of graphs, and you want the receiver to hone in on a specific section of the report. In a Notebook report on Kyso, click the `"link"` icon to the right of a cell. This will open up a new URL to that specific cell. {% hint style="info" %} Note that you need to hover the mouse over a cell to see action icons, like "New Task" and "Open Cell Link." {% endhint %}

Report Snippets

**Note the URL in the image above!** Users can open up the rest of the notebook in the top right of the report! # Business Notifications Communicate events to everyone on Slack or Teams so everyone is kept up to date. ## Email Notifications Kyso is configured by default to send users email notifications for MANY different event actions. As a user, you can configure which events you would like to receive emails for & which not. To do this, go to **Settings**.

Kyso Settings

Click on **Notifications**. Note that you can configure your preferences globally or by Organisation & Channel.

Notification Events

## Centralised Communications As an account admin you can configure to which email Kyso notifications will be sent. Simply head to **Settings**, go to your Organisation of choice, and to the **Notifications** tab. This just means that all Kyso event communications will flow through a specific email address that you specify, instead of going to all users in the Organisation. Select to **Enable Centralised Communications** and add the email. Don't forget to hit **Save**!

Centralised Communications

## Business Comms Notifications {% hint style="danger" %} Current limitation: all event notifications are sent to Slack and Teams from Kyso. There is no option (currently) to customise your own communications preferences. This is only possible for email communications. {% endhint %} {% content-ref url="/pages/vnHHeYyzXlwUDHBCwCkW" %} [Slack](/collaboration/business-notifications/slack) {% endcontent-ref %} {% content-ref url="/pages/7RzQydsUtUbrSq49oDJ8" %} [Microsoft Teams](/collaboration/business-notifications/microsoft-teams) {% endcontent-ref %} # Slack Keep Kyso and Slack in sync so the team doesn't miss anything {% hint style="danger" %} Current limitation: all event notifications are sent to Slack and Teams from Kyso. There is no option (currently) to customise your own communications preferences. This is only possible for email communications. {% endhint %} Kyso can be integrated easily with Slack, which will receive notifications when something happens in your organization or channel, for example, when a new report is created, or a new comment is written, just follow these steps: ## 1. Create a Slack App Open Slack and choose the option **Apps**, under the menu *More.*

Slack Apps

Click on **App Directory** button, located at the top right of your screen.

Slack's App Directoy

Then, click on **Build**, located at the top right of your screen.

Slack Apps

Go to **Your Apps** in the top right of your screen, go to **Manage your apps** and click **Create an app**

Your Slack Apps

Click **Create an app**

Create an App

Select the option **From scratch**

New App from Scratch

Choose a **name** and select the desired **workspace**

Choose an App Name

Click on the section **Permissions.**

App Permissions

Scroll down until the section **Scopes** and add the scope **chat:write** to the **Bot Token Scopes.**

App Tokens

Scroll up to the top of the same page and click on the button **Install to Workspace.**

Auth Tokens

Copy the generated **Bot User OAuth Token** and save it for later use.

Auth Tokens

The Slack App configuration is finished, in the next section you will configure Kyso to use this Slack App. ## Configure Kyso Slack notifications can be configured at different levels: * **Organization level**. All the channels under an organization will *inherit* this configuration. That means, all the notifications related to that organization will use the same Slack channel. *Requires organization admin role*. * **Channel level**. Every channel can define their own slack notifications. If there is a configuration at organization level, that configuration will be *overwritten* by the channel configuration. Using this, you can define specific configuration per channel. Requires Full Access role. #### Configure Slack Notifications at Organization Level Login into Kyso, click on your avatar and select **Settings**.

Kyso Settings

Choose the desired organization in the left menu, select the tab **Notifications** and copy & paste the **Bot User OAuth Token** in the form *Slack Token*. Also, specify the **destination channel** in the input *Slack Channel*.
Open Slack, click on the **destination channel** and open the configuration by clicking on the title of the channel, at the top of the page.

Channel Configuration

Select the tab **Integrations** and click on the button **Add an App**.

Adding an Integration

Write the name of your app and click on the button **Add**.

Add the Kyso App

Well done! Your organization is now configured with Slack, to test it out just create a new report in your organization and check that you are receiving the notifications in the configured channel at Slack.

Kyso Notifications

## Configure Slack Notifications at the Channel Level The configuration process is exactly the same, but in the **Channel Configuration** page on your Kyso settings.

Kyso Channel Settings

Take into account that the **slack token** only can be defined at the **Organization level**, for security reasons. # Microsoft Teams Keep Kyso and Teams in sync so the team doesn't miss anything {% hint style="danger" %} Current limitation: all event notifications are sent to Slack and Teams from Kyso. There is no option (currently) to customise your own communications preferences. This is only possible for email communications. {% endhint %} Below is a simple follow-along demo video to help you get started with our Microsoft Teams integration: {% embed url="" %} Kyso & Microsoft Teams {% endembed %} ## 1. Select Team & Channel To connect Microsoft Teams to Kyso, all we need is an incoming webhook URL from Teams. So in our office account, go to Teams, select the team in question and select or create the channel you want to publish this integration to. ## 2. Create the webhook URL Next, we need to go to the channel connectors and add an incoming web hook. So click **configure** here and give the webhook a name and click **Create**. ## 3. Add the webhook to Kyso This will generate the webhook URL, copy this to the clipboard. Back on Kyso, let's go to the settings of a specific organization, to notifications and add the URL to the Teams setting and hit **save**. Now, every time some action is generated on Kyso, a notification is pushed to our new Teams channel. # Themes & Styling Learn how to customise the look and feel of your team's Kyso deployment At Kyso, we know that aesthetics matter, especially when working in on-premise installation! {% hint style="success" %} In a default installation of Kyso, **no theme is installed**. If you want to start your customisation, please download the following file. {% endhint %} {% file src="/files/fOgwiWWyCywKbnWxGyrA" %} Basic Getting Started Theme {% endfile %} Using the Kyso CLI, we can use a little bit of CSS magic! A **global administrator** can edit the global styles of Kyso. For example, you can add the company's logo, or change the colors to match the corporate identity. Just type in your shell the following commands to get started: ```sh $ kyso theme Uploads the to the given folder replacing its previous contents. USAGE $ kyso theme COMMAND COMMANDS theme add Uploads the to the given folder replacing its previous contents. theme del Removes the folder. theme get Downloads the contents of the theme_name folder and stores it on the given zip_file or on the theme_name.zip file on the current directory if no zip_file is provided. theme list List all available themes theme set Set the as default. ``` {% hint style="info" %} The Global Admin will usually be the person who set up the Kyso installation. You can configure this or add more Global Admins in the install scripts. {% endhint %} # Permissions System Learn about Kyso's Enterprise permissions system here! ## Introduction The following document describes how the permission system works at Kyso, which is highly customisable, in which we can create roles with fine-grained permissions to configure how your users can behave on the platform. ## Platform Roles Kyso has default access controls on deployment. Below is a list of the predefined roles called **platform roles**. These roles are the most common ones, and are “self-explanatory”. The roles are the following: * Platform / Account Admin * Organization Admin * Channel / Team Admin * Channel Contributor * Channel Reader {% hint style="info" %} Channels and Teams mean the same thing on Kyso so might be used interchangeably across the public documentation and Git repositories. {% endhint %} ## Role Access Levels ### Table Legend * 🎷 -> Only for the entities that the user owns or belongs to (comments, reports, repos, organizations, etc.). That means that users can’t perform these actions in other channels/discussions/reports/etc. in which he/she is not a member of nor owns it. * 🔓 -> Only public channels accessed directly with the URL. ![Legend Part 1](/files/p0gQ7vLg5zrVVEblsdGU) ![Legend Part 2](/files/vndWntLhd4AQR7llvjBG) ![Legend Part 3](/files/bDttbaFTiy4qVcAjY84y) ![Legend Part 4](/files/z8j60hi557NhciBvTfJf) ## Platform Organisational Structure ### 1. Organizations An **organization** is a management unit at Kyso to manage the permissions and the behaviour of users that belong to it. Consider that: * Every user belongs to at least one organization * Every time a new user is created at Kyso, a new organization “{{username}}’s Workspace” is created (i.e. this is their personal workspace) * A user can belong to more than one organization. ### 2. Channels / Teams A **channel** is a management unit at Kyso, to manage the visibility and the behaviour of the users. Consider that: * Every user belongs to at least one channel * Every time a new user is created, a new channel “{{usernames}}’ Private Team” is created and linked to “{{usernames}}’s Workspace” * A user can belong to more than one channel * A channel can be: * **Public**: Every user on the company's Kyso account can access that channel. * **Protected**: Only users that belong to the organization that owns the channel can access that channel. * **Private**: Only users with a specific invitation can access that channel ### 3. Reports A **report** is an instance of an imported notebook (Jupyter, etc.) or other file type. All the data, comments, collaboration, etc. is done inside a report. Consider that: * Every report belongs to a channel. ## Custom Roles The permission system allows the creation of new roles, with a specific bunch of permissions, in organization and channel scopes. {% hint style="info" %} Contact our Support team for more info on this! {% endhint %} That means: * An organization admin can create a new role that is only valid inside its organization * Then, the users that belong to that organization can be configured to use: * All the Platform Roles * The custom roles of that organization * A channel admin can create a new role that is only in valid inside its channel. * Then the users that belong to that channel can be configured to use: * All the Platform Roles * The custom roles of the organization that owns the channel (if there is one) * The custom roles of that channel ## Authorization Hierarchy ```md . └── Platform Roles └── Organization Roles └── Channel/Team Roles ``` # Managing Access How administrators at all levels can manage access to your company's research. Navigate to your avatar in the top right-hand corner and go to *Settings.*

Kyso Settings

This is where you can manage your personal settings and company settings (to the extent that you have some admin control).

Kyso Organisation & Channel Settings

You can see all the organisations and channels that you have access to, as well as all the **Members** that belong to each Organisation and Channel. Select an Organisation in the sidebar to see all the channels that you have access to **in that Organisation.** Click on a Channel in the *Channels* tab to reveal all of the teams that you have access to. Click into any Channel to reveal the team-specific settings:

Channel Settings

{% hint style="warning" %} Note that Kyso Organisations are public (on the network) by default, so users without explicit access to an Organisation or those who have been removed will still have access to that Organisation (to all public & protected channel content). {% endhint %} ## Administrative Control & Platform Roles There are a small set of roles predefined at Kyso that can be used across the entire application. These roles are just a specific set of permissions grouped under the “umbrella” of a role. {% hint style="info" %} Channels and Teams mean the same thing on Kyso so might be used interchangeably across the public documentation and Git repositories. {% endhint %} These are the Platform Roles available: * **Platform / Account Admin** (platform-admin). Full access to every single entity at Kyso. Also known as the **Global Admin**. Full control without restriction. * **Organization Admin** (organization-admin). Full access but restricted to a specific organization. While the Platform Admin can make changes in every organization, an organization admin only can perform that changes inside his organization. An organization admin can’t: * Create new organizations. This should be done by the **Platform Admin**. * **Team / Channel Admin** (team-admin). Full access, but restricted to a specific channel inside an organization. A team admin cannot: * Create new teams. This should be done by the Organization Admin. * **Team Contributor** (team-contributor). Allowed to create and edit content inside a channel (comments, reports, etc.) * **Team Reader** (team-reader). Only allowed to read content inside a channel. Consider that: * A user can play different roles across different Organizations and Channels. * For example, the user “” can be an **Organization Admin** in the organization “Kyso” but a **Team Reader** in the organization “Acme”. * That’s also true for teams / channels, the user “” can be a **Team Admin** in the channel “my-awesome-team” in the organization “Kyso” but a **Team Reader** in the channel “another-awesome-team” in the same organization. # SSO Configuration # How to manage adoption Learn our best practices on segmenting users and content, and scaling adoption of the platform across the organisation. All organisations - and especially larger companies with many different teams and departments - will need to conceptualise how to segment their research and user groups early. The Kyso platform is structured as below: * Company Account * **Organizations**: Think of these as different workspaces, or departments within the business. * **Channels (Teams)**: Each organization can have multiple channels to organise users and reports. Think of these as similar to Slack channels. * **Reports**: Every time a report is published to Kyso, it is published to a specific Team, within an Organization. To ensure a successful publication, ensure that a report's metadata contains the following variables to correctly categorise it: * `organization: ` * `channel (or team): ` ## Channel & Report Visibility Settings You can review how access permissions work in the following document: {% content-ref url="/pages/vxPiOwAgmzahQZSaa0Do" %} [Permissions System](/settings-and-administration/permissions-system) {% endcontent-ref %} Channels have different privacy settings: * **Private:** only direct members of this channel have access to this team's content. * **Protected:** only members within the organization can access this channel's content. * **Public:** any member of any organization can access this channel's content. * Reports in this channel can also be viewed by external users with no Kyso account by sharing a report's shareable link. Note that users will need authorised access to your company's Kyso deployment (for example, if you have deployed a Kyso self-hosted behind your company's firewall. Users can directly browse through channels **to which they have access.** # Driving Internal Engagement How can you ensure on-boarding success and long-term value creation for all employees? ## Scaling Adoption We have found that, especially for larger organisations, the big-bang approach - opening up the platform to everyone from the beginning - will not lead to success. Adoption at scale happens gradually, it is best to segment your user base and open up access to the tool using the tiered approach laid out below.

The Kyso Adoption Rollout

### Your core creators Start with your core creators - these are typically your more technical users that work with notebooks. **Data teams, engineers, bio-statisticians, scientists, etc.** Kyso is not like other community-driven platforms. Twitter, for example, is only useful if there are many users. With Kyso, however, there is a high-value use-case even for an initial small segment of users who can: * Publish and share reports with other users: * Review each other's work. * Collaborate over global report & inline commenting. * This creates more transparency and reduces duplication of work across teams. These conversations are indexed for future search. * You can trust this group to contribute, add & engage with content, and discover ways of using the platform. They will also set the cultural norms of the hub and the bar for quality of contribution. * They will be the early and most active contributors that build momentum for adoption and pull other users in. ### Management and business stakeholders Once you have full adoption within a small active group, posting and collaborating on reports, you have succeeded in *seeding* content and have a growing base of helpful, relevant conversations. Now it's time to scale to management teams and other core business stakeholders who usually have the longest email chains - siloed conversations - with the technical users from the first group. You want to break these chains, and allow them to ask questions openly, track projects and have access to a single reference point. This group of users can now openly ask questions, make requests, and track projects from inception to the deliverable, all on one single platform. ### Knowledge consumers This is the last level where you now release the platform to the organization at large, to democratise access to all data-based reporting and discussion. This is when organic growth becomes viral. If the first two stages have been successful, the value of the network will grow with each additional user. By adding a large number of people onto the knowledge base, the network can generate an enormous amount of added value. The whole business can now use Kyso as a consistent and verifiable source of information where anyone can search and discover knowledge that may be relevant to their respective role in the organization. Now more and more users will begin to organically adopt the platform as they see value in participating for themselves. The whole company culture shifts & begins to embrace increased openness and information sharing. ## Tips & Tricks for Success Kyso is now your central knowledge hub for all data-based information sharing and collaboration around the business. It encourages multi-channel communication flows, bringing everyone into the conversation, creating **empowerment loops**. There is a network within **your organisation** and such a tool becomes more valuable the more it is used. Here are 4 things to consider when starting out: ### 1. How to create **value up front?** It is much easier for people to understand the value of the tool & to get involved when they sign up if there is already a clear starting point. **Remember that Kyso is so much more than just a reporting tool.** Knowledge doesn't come from analysis & reporting alone - but also from the collaboration that these resources inspire. Have a #welcome or #general channel that introduces new users to the platform. You can pin reports with demo videos, tutorials and documentation. Use one of these reports to show the user how they are expected to use Kyso. For example, depending on what team they are from, where should they post their work? Publish existing and ongoing projects to demo upfront the type of content users can expect to discover of Kyso moving forward. Make Kyso core to the publishing process for ongoing & future projects. You can tag users on report comments to kick off discussions. This will help different teams connect and avoid any duplication of work. These users can migrate ongoing discussions currently siloed in private Slack or email channels, with all accompanying files and graphics. ### 2. What is the engagement pyramid? Remember that not everyone will contribute and that is okay and expected. Typically, "participation" for larger organisations follows an engagement pyramid pattern: * 1-10% will be very active, not only posting reports but also interacting with other contributors on the platform. * 10-30% will be moderately active, sporadically creating reports, but engaging more on content created by others, asking & answering questions, up-voting, sharing, etc. * The rest are your knowledge consumers - readers only. These are the users that use Kyso as their daily source of information relevant to their respective roles around the business. * Remember that the number of active contributors **is not** the ultimate success criteria, it is the learning, collaboration and application of insights that happens on an ongoing basis. ### 3. How to organise your content? * The core benefit to leveraging a tool like Kyso is the consolidation of all data-based content, which is otherwise siloed in individual conversation channels. * Effect a canonical publishing structure, leveraging the entirety of Kyso's hierarchical structure and content metadata to organise content: * Segment your user groups and research into Organisations and Channels. How you do this will vary business to business. See our current recommended options on how to do this, based on research to date: {% content-ref url="/pages/rVu18Poia2fQHLvHKhY0" %} [Advice for Large Companies](/guides/how-to-manage-adoption/advice-for-large-companies) {% endcontent-ref %} * Construct a guideline taxonomy for metadata creation: * Ensure your content creators publish their research into the appropriate Organization and Channel. * Make sure they are applying relevant tags only (you can even control how/what tags are created if you like). * This will ensure all content is relevant, current, and verifiable. Any available information within a project report will be easy to search and discover quickly. * Reports can become comprehensive resources by consolidating information into one topic, which links out to other important sources. * It is in this way that Kyso will become the default "open tab" in browsers across the entire organisation. If users are visiting the platform every week, it means they see it as a persistent, trustworthy and valuable source of information. ### 4. How to recognise and promote quality contributions? * Appreciate all good content and active contributors. * Reward and incentivise contributions through recognition. For example: * Up-voting/pinning valuable reports and and discussions. * Weekly or monthly shout-outs. * Make participation as important as publishing content. Responding to reports and comments immediately provides value back to multiples of other users. Every action is a feedback mechanism that creates added value to the rest of your user-base, such as voting or commenting on content. * You will know Kyso is valuable to the organization as a resource when the up-vote, view and comment counters are ticking up over time. **Employing these strategies will allow you to build trust in the platform and content, create more value for users over time, and spark organic growth so that the knowledge-base becomes self-sustaining.** # Advice for Large Companies How hundreds, or thousands, can communicate better A large-scale team has particular needs and challenges because your work, functions, and interactions can be very complex. Here are some options that have worked for existing clients, while maintaining as **flat a structure as possible** (important for discoverability). ## Option 1: Unlimited Channels \[Our Recommendation] A lot of people's default reaction to this option would be to think that **too many channels can become mess**. From our research, there isn't such a thing as too many channels, as long as they are organised effectively. 1. Keep the one Organization for all company teams, maintaining a very flat structure. 2. No limit on number of channels, but segment by channel type - team, event, project, etc.. 3. Follow a strict naming convention (see below) 4. Tags are used for “topics” rather than areas, teams or projects #### Channel breakdown example * One main channel for each team/department * For research that can be shared with everyone, having the one “main” channel can work towards “department branding” * \#dept-data-science \[public] * \#dept-sales \[public] * \#dept-engineering \[public] * Have a channel for onboarding reports. If a user is unsure where to publish their first report, they can do so here, and the channel admin can move it accordingly. * “#welcome” \[public] * Have channels specifically for events * “#event-hackathon-2023” \[public] * And project-specific channels for sub-sets of users, for private & task-related research. * “#project-external-collaboration” \[**private**] * “#project-sales-projectName” \[**private**] **Non-permanent channels should be archived when not in use/project is finished.** Kyso will maintain everything from those channels, and users will still be able to search for them. Any project-specific channels for private work and/or specific sub-segments of users need to be made private. In this way, users will only see a list of public channels and the private channels to which they have access. So no one user should see 00s of channels. ## Option 2: **More “Public” Organisations** Another option could be to add more entities to the top layer of Kyso's hierarchy. Maybe **better for categorisation**, but does **add more clicks to a user's browsing experience**! 1. Each (big) team or department has their own organization on Kyso. 1. E.g. Acme, R\&D 2. Organisations would be public by default. 3. Channels represent topics 1. E.g. #genomeA 4. Tags are used for further categorisation within channels/teams 1. E.g. #test1 Note that currently users who are NOT explicit members of an organisation will be able to find an organisation’s content only when using the global search function. This is because you (currently) only see organisations on the homepage of which you are already a member. If you want to go with Option 2, we can customise your deployment and list all Organisations on the homepage by default. Users could click into any Organisation, which are all Public by default How this would work: Explicit members of the organisation see all channels - public and protected. Some members will also see private channels that they have access to. Non-members will only see public channels. ## Switching between options If, at any time, you decide that you'd like to switch how you segment users and content, but already have 00s, if not 000s of reports and users, don't worry. We will work closely with you to make the transition as smooth as possible :) # R Users & R Markdown Kyso is also designed for R users to also be able to publish their data-science reports. When posting R markdown (`.rmd`, or `.Rmd` files) to Kyso you will need to convert them to normal markdown (`.md`) files before they hit Kyso. That can be before you upload them, or before you commit them to your Github repository. When you convert Rmd to normal markdown it will create run the code in the Rmd file, generate any images and tables needed, create a markdown file and an assets folder for any image files. You can them upload the markdown and assts folder to Kyso. ## Converting R to other formats ### Installing knitr Knitr is the package that we will use to convert the Rmd files to markdown in order to upload to Kyso, lets install it first using inside your R console: ``` install.packages("knitr") ``` ### Using the command line In the directory of your .Rmd file just use the following command: ``` Rscript -e "library(knitr); knit('notebook.Rmd', 'notebook.md')" ``` ### Using R Studio Make sure you set the working directory to the location of your .Rmd files. ![R Studio Interface](/files/-Lq0LHyNzHA0ugLpx9zk) Once you have your working directory set go to your R console inside R studio and enter the following commands, first to load knitr and then to convert the notebook: ``` library(knitr) knit('notebook.Rmd', 'notebook.md') ``` ![Compiling R markdown to markdown](/files/-Lq0LrqnRxGAwSclvRiQ) {% hint style="info" %} When publishing to Kyso, be sure to select the generated .md file as the main file, not the .Rmd {% endhint %} ## Compiling R to HTML Kyso now also supports HTML! This means R users are also able to compile their scripts to HTML. This will benefit those who like to work with interactive graphing libraries like plotly and bokeh. Using the above example: ``` library(knitr) knit('notebook.Rmd', 'notebook.html') ``` And then follow the link below to learn how to ensure a successful upload of HTML to Kyso: {% content-ref url="/pages/qRkNxsUCIee3Clo06K1T" %} [HTML](/kysos-renderer/html) {% endcontent-ref %} # Writing a good data-science report This guide will go through how you can write a really good data science article that will appeal to your readers. This is applicable to both public and private posts. At Kyso we're building a platform upon which knowledge gained from data can be communicated to everyone on a team or within a company, in a way that is understandable to as many people as possible, for both technical and non-technical readers alike. This guide will go through how you can write a really good data science article that will appeal to your readers. ## **Have a clear objective** Explain clearly at the beginning what you’re article is going to be about and the data you are using. Provide some background to the topic in question if necessary & explain why you are writing the post. ## **Structure your writing** Determine the logical structure of your argument. Have a beginning, middle, and end. Provide a table of contents, use headings and subheadings accordingly, which gives readers an overview and will help orientate them as they read through the post — this is particularly important if your content is complex. Aim for a logical flow throughout, with appropriate sections, paragraphs, and sentences. Keep sentences short and straight-forward. It’s best to address only one concept in each paragraph, which can involve the main insight with supporting information, such that the reader’s attention is immediately focused on what is most important. ## **Plotting libraries** There are many visualisation tools available to you. For static plotting or for very unique or customised plots, where you may need to build your own solution, matplotlib and seaborn are your answers. However, if you really want to tell a story and allow the reader to immerse themselves in your analysis, interactivity is the way to go. Your two main options are Bokeh and plotly. Both are really powerful declarative libraries and worth considering. Our team at Kyso tends to use plotly as we feel the syntax is more intuitive than Bokeh, which means plots are quicker to make and you have to spend less time searching for the various functionalities. The plotly documentation is also much more comprehensive than that of Bokeh & there are plenty of examples of every graph you can think of. Regardless of the one you choose, we recommend picking the one library and sticking with it until there’s a compelling reason to switch. Altair is another (newer) declarative, lightweight, plotting library, and merits consideration. Check out its gallery [here](https://altair-viz.github.io/gallery/index.html). ## **Plotting multiple charts** As discussed above, plotly has become the leader for interactive visualisations, which beautifully render on Kyso. Its efficiency and functionality goes above and beyond any other library out there. However, because plotly stores the data for each plot generated in the user’s browser session and renders every interactive data point, it can really slow down the load time of the post if you are working with multiple plots or with a very large dataset, which negatively impacts the reader’s experience. If you are generating a lot of graphs or are working with very large datasets but wish to retain the interactivity, use Bokeh or Altair instead. ## **Document your charts!!!** When plotting make sure to have explanatory text above or below the chart — explain to the reader what they are looking at, and walk them through the insights and conclusions drawn from each visualisation. ## **Labels and high-resolution images** Label everything in your graphs, including axes and color scales. Create a legend if necessary. Your plots should be high-resolution, automatic for newer libraries like plotly and Bokeh, but if you’re using matplotlib or seaborn, execute the following code at the beginning of your notebook: ``` %config InlineBackend.figure_format = ‘retina’ ``` to configure the Jupyter backend to apply retina display mode. ## **Show code by default if a guide or tutorial** If you are writing a guide or a particularly technical post, in which you are constantly referring to the code, you should show the code cells by default. On Kyso, you can toggle the code visibility in the top right-hand corner of the post — you can show code input or both input & output. ## **The Conclusion** Don’t just taper off at the end of the article or finish with the final point in the main body. Give the reader a quick summary of what they have learned, explain how the insights gained impact the business and how your team members should apply this knowledge in their own work. For fellow creators or other members of the data team, have a call to action — perhaps a recommendation for extending the analysis. ## **Before posting** Before you publish your work, return to the basic questions you wanted to answer at the beginning. Have you answered all of them? Have you done your best to make it as easy as possible for your readers to understand your work? Be sure to add an appropriate title, description, tags, and preview image in the settings tab of your post or in the *kyso.yaml* file. This is important for organising the team’s work on Kyso. For open-source writers, it helps when sharing and promoting your work around the web — presentation is key.