Skip to main content

Working with docker metaframes

Now that you know how to create simple metapages, from adding metaframes and constructing data connections between them, to uploading and displaying images and running python with pyodide and displaying its output in a linked metaframe, we will now learn how to use one of the most powerful tools in metapages, the Docker metaframe. In basic terms, Docker is a technology that allows you to create and run virtual computer systems within any other system. This is powerful because in this system, called a container, you can install exactly the software needed to run code, and the same docker container is identical no matter what system it is run on, so your code will always work, no matter where or when you run it. This powerful technology can be leveraged in metaframes with custom docker containers or configurations, allowing you to run virtually any code you can imagine. The applications are endless! To get you started with Docker in metaframes, let’s go through the simple example of creating a word cloud with python.

  • Step 1: As done in the previous tutorials, create a new metapage. We will call ours “Word Cloud with Docker”

  • Step 2: We will use four metaframes to construct this metapage. Two “Text Editor” metaframes, one “Show Image”, and one “Docker” metaframe. To add these, enter Edit mode, use the search bar to find them (the official versions should have the “base” tag), and add them to the metapage with the Add button. Feel free to resize and rearrange the metaframes before exiting Edit mode according to your preferences.

  • Step 3: We will use the ”Text Editor” metaframe for the text input to the word cloud and “Text Editor-1” for the python code to create the word cloud. The “Docker” metaframe will run the python code in a docker container using the input text and produce a word cloud image which will be displayed in the “Show Image” metaframe. Let’s now create the necessary data connections between the metaframes. To do this, click on “Data flow”, and then follow the steps below.

  • Step 3a: We first want to connect the input text from “Text Editor” to “Docker”. To do this, first click on Docker to open the connection dialog box. We will provide the text to Docker as an input file called “input.txt”. Under Source Metaframe click “Text Editor” to select it. Then under outputs click on the box labeled “text”. Under inputs, click Add, then type “input.txt” (without the quotes) followed by Enter. Click OK to proceed.

  • Step 3b: Repeat the procedure from 3a to provide the text from “Text Editor-1” to docker as a file called “generate_wordcloud.py”. It should look like the image below when finished.

  • Step 3c: Finally, connect the output image from Docker to the “Show Image” metaframe. Click on “Show Image”, ensure “Docker” is listed as the Source Metaframe, then add “wordcloud.png” as the output as input. It should now look like this.

  • Step 4 - Add the text content: Now that we’ve added our metaframes and connected them together let’s begin adding the content and configuring our metaframes. First we need some content to create our word cloud. This can be whatever you want, but for this example we’ve taken the first few paragraphs from the Docker Wikipedia page. First return to View mode, and then add your content to the “Text Editor” metaframe.

  • Step 5 - Add the python code: To provide the python code that the docker container will run, you can simply copy the contents of the code block below and paste it into the “Text Editor-1” metaframe. You may optionally want to adjust the options of this editor to use a dark theme or provide python keyword coloring by selecting python under the Editor code mode dropdown.
import os
from wordcloud import WordCloud
import matplotlib.pyplot as plt

# Read the text from the file
with open('/inputs/input.txt', 'r') as file:
    text = file.read()

# Generate the word cloud
wordcloud = WordCloud(width=800, height=400, background_color='white').generate(text)

# Save the word cloud as an image file
wordcloud.to_file('/outputs/wordcloud.png')

print("Word cloud image generated as 'wordcloud.png'")

Step 6 - Configure Docker: Finally, now that we’ve laid all the groundwork, we will configure our Docker metaframe in order to actually create our word cloud. In configuring the Docker metaframe it is important to know that any inputs can be accessed in the /inputs/ folder and any outputs need to be routed to the /outputs/ folder. You may have already noticed this reflected in the python code from the last step. There are two main things that we need to do to configure the Docker metaframe. You need to specify how to set up the container, and then specify what it will run. To do this, click on the gear icon at the top right of the metaframe and follow the steps below.

Step 6a - Configure Image: Access the Docker Image configuration by clicking on the “Docker Image” tab. There are two main ways to specify how to create your docker image. You can use an existing published image by specifying the Docker Hub container identifier under “Use Existing Image” or you can create a custom image with “Build Image from Repo”. We will do the latter. After selecting “Build Image from Repo”, click Edit next to Dockerfile. You can then paste in the code below and click “Save”.

# Use an official Python runtime as the base image
FROM python:3.10-slim

# Install system dependencies required for wordcloud (gcc and font libraries)
RUN apt-get update && apt-get install -y \
    gcc \
    libfreetype6-dev \
    libpng-dev \
    && rm -rf /var/lib/apt/lists/*

# Install Python dependencies directly
RUN pip install wordcloud matplotlib

You can leave all the other Docker Image options blank and click Save again.

Note: A Dockerfile includes all the commands which will be run in setting up the Docker container. In our Dockerfile we use a lightweight version of python as the base image, install some system dependencies, and then finally install the python packages we will use.

Step 6b - Configure Job: In the last step we specified what to install in our container and now we need to tell it what to run. Go to the Configure Job tab and under Command type “python /inputs/generate_wordcloud.py”. This will instruct Docker to use python to run the generate_wordcloud.py script file in the /inputs/ folder. All other boxes here can be left blank. Click Save.

Step 7 - Run the Job: Everything is now set up. Just click “Run Job” in the Docker metaframe to build the Docker container, run the python code, and generate your word cloud.

Congratulations! You now have a very powerful tool at your disposal. We’ve gone through a simple example here, but you can adapt the principles learned here to run any code you like inside the Docker metaframe. This can be leveraged to create arbitrarily complex pipelines for data processing, analysis, and visualization for a wide range of applications in drug discovery, computational biology, bioinformatics, machine learning, and beyond. The only limit is your imagination.

This concludes our Getting Started tutorials on metapages, but there are many more applications to learn and discover. To learn more and get inspired, please explore the application-specific Examples and Walkthroughs sections of the docs as well as our Gallery of Metapages and Metaframes.

Solution