Artifacts and files#
PlayMolecule has two related but distinct concepts for “things that live inside an app’s container and can be used by it”: files and artifacts. They share the same underlying class hierarchy but serve different purposes.
Files: the raw inventory#
Every app declares a files block in its manifest mapping logical paths to in-container paths:
"files": {
"tests/3ptb.pdb": "/app/files/tests/3ptb.pdb",
"tests/web_content.pickle": "/app/files/tests/web_content.pickle"
}
These are exposed as app.files, a dict of file handles. You don’t usually touch this directly. It’s used internally to:
Resolve test-config paths to actual file handles (
tests/3ptb.pdb→ the bundled PDB).Resolve
artifactsentries (next section).
Artifacts: the curated, callable surface#
The artifacts block (also accepted as the older synonym datasets) declares which files are meant to be used as inputs:
"artifacts": [
{ "name": "default", "path": "datasets/model_98acc.ckpt", "description": "DeepSite final model" }
]
These appear as attributes on app.artifacts:
from playmolecule.apps import deepsite
deepsite.artifacts.default # a callable file handle
deepsite.artifacts.default.path # path inside the container
deepsite.artifacts.default.download("./local-copy")
You pass deepsite.artifacts.default directly as a function argument; PlayMolecule resolves it to the right path depending on the active execution backend (local mount, Docker bind, HTTP fetch).
Files vs. artifacts#
Aspect |
|
|
|---|---|---|
Source |
|
|
Access |
dict keyed by logical path |
attribute access by curated name |
Purpose |
wiring (tests, internal resolution) |
user-facing; pass into app calls |
Has |
Logical path |
Curated short name (no dots; must start with a letter) |
Has description |
Usually no |
Yes (from manifest) |
Downloadable? |
Yes ( |
Yes ( |
artifacts is files filtered down to the entries someone took the trouble to curate and name. Reach for artifacts unless you know you need the raw files dict.
Backend-aware file handles#
A file handle knows how to fetch its content for the active backend:
local registry: plain filesystem path;
.download()is a copy.Docker registry (Docker runtime):
.download()shells out todocker cp.Docker registry (Apptainer runtime):
.download()runsapptainer execagainst the cached SIF and copies out.HTTP backend:
.download()issues an authenticated GET to the backend.
You don’t pick which one you got; the handle does the right thing for the current registry/runtime, which is why example code can use app.artifacts.Foo uniformly across installations.
When you’d actually use .download()#
The download() path is for outside-the-app consumers, say an analysis in your own notebook that uses the same reference data as the app. For arguments to the app, just pass the handle directly; don’t download first.
Artifacts outside the image#
Some apps need files too large to ship in their image, multi-GB models for example. Baking them in would make every image version that much larger, and an Apptainer install keeps a full SIF per version. These live once per install instead, outside any image:
On a machine or cluster: in
PM_ARTIFACTS_ROOT, one folder per app (<root>/<app>/...), filled by an admin.On a Modal-backed server: in the Modal volume named by
queue.modal.artifacts_volume, with the same one-folder-per-app layout.
They are the opposite of the manifest artifacts above: for the app’s own use,
never shown to, downloaded by or passed in by users. Each run gets only its
app’s folder, read-only, mounted at a path drawn fresh for that run, so no
input or symlink can be aimed at it in advance. The app reads it from the
PM_ARTIFACTS_DIR environment variable:
import os
model = os.path.join(os.environ["PM_ARTIFACTS_DIR"], "esm2", "v1", "model.pt")
To let users choose between several models, give the app a parameter naming the model (an enum, say) and map it to a path inside the app. Never take the path itself as an input. Keep one subfolder per version and never overwrite it, so a past run still means what it did.
Read-only mounts stop apps from writing these files. They cannot stop an app that copies its model into its outputs, or that runs code the user supplies, so an app given a folder must do neither.
Gotchas#
Two artifacts in the same app cannot share a name. If they do, the loader overwrites silently.
download()to a path that already exists will overwrite a file or wipe-and-recreate a directory. There is no “keep existing” mode.For Docker / Apptainer files,
download()shells out, which is slow for many small files. Prefer a singledownload()of a directory over a loop of per-file downloads.