📓 Jupyter Notebook Workflow Tutorial

BAM–IWM PyIron Knowledge Graph
Integration Workflow

Step-by-step tutorial for material parameter identification with the SDCHABOX creep model, automated knowledge-graph publication to the MaterialDigital data portal, and SPARQL-based Material Card Pass generation.

🏛 BAM Federal Institute for Materials Research 🔬 Fraunhofer IWM · BWMD Ontology ⚙️ pyiron-workflow · PyironFlow · CKAN · Fuseki 🧪 Material: IN718 superalloy · Model: SDCHABOX
★

Introduction

This tutorial documents the demo_gui_use_case_trell.ipynb notebook, which implements a complete end-to-end pipeline:

📌

Calibration workflow

Original BAM sections — three-stage optimisation of SDCHABOX creep parameters for IN718.

🌐

Knowledge graph

New IWM integration — all results published to the BWMD ontology and indexed in Fuseki.

🗂

Material Card Pass

Auto-generated HTML report from 6 SPARQL queries, published to dataportal.material-digital.de.

0

Environment Setup

⚠ Critical — use the correct conda environment

JupyterLab and the kernel must both come from pyiron_env. Mixing environments causes widget version mismatches and silent rendering failures.

Required packages

Python
3.12.13
JupyterLab
4.5.8
anywidget
0.11.0
pyironflow
0.0.21
pyiron-workflow
≥ 0.17.0
openpyxl
any
requests
any
rdflib
any
pandas
≥ 2.0

Step-by-step setup

1

Create the conda environment

# Use Miniforge Prompt or CMD (not PowerShell)
conda create -n pyiron_env python=3.12
conda activate pyiron_env
pip install pyiron-workflow pyironflow openpyxl requests rdflib pandas
2

Launch JupyterLab from within the environment

conda activate pyiron_env
jupyter lab "C:\path\to\Pyiron_Windows_06_10_2026\demo_gui_use_case_trell.ipynb"

Always launch from Miniforge Prompt or CMD. PowerShell does not honour conda activate for PATH switching, which causes the wrong anywidget version to be loaded.

3

Generate your CKAN API token

Required for the upload steps in Sections 4–6.

  1. Log in to dataportal.material-digital.de
  2. Click your username → My Profile
  3. Scroll to API Tokens → click Create API Token
  4. Copy the generated token and paste it into ckan_upload_nodes.py → NEW_CKAN_TOKEN
4

Prepare configuration files

FilePurpose
Main.txtParameter names, bounds, initial values, MATERIAL_ID, MODEL_NAME, metadata
Input.pyOptimizer settings, material_id/model_name variables, file path configuration
sim/SDCHABOX.mat.tmplParameter template for the simulator
Validation/Validation.txtValidation test configuration including MATERIAL_ID and MODEL_NAME
1

Setup Project original BAM

The SetupProject node prepares all simulation inputs and pre-processes experimental data before the optimizer can run.

Main.txt exp/*.exp files SDCHABOX.mat.tmpl SetupProject node PyIron @as_function_node sim/*.inp & *.uma exp/*.lexp (log-transformed) project_dir (output channel)

Notebook cell

import importlib
import setup_project_nodes as spn
from pyiron_workflow import Workflow
from pyironflow import PyironFlow

wf = Workflow("setup", delete_existing_savefiles=True)
wf.Setup = spn.SetupProject(
    project_dir = 'C:/path/to/Pyiron_Windows_06_10_2026'
)
wf.run()  # or: pf = PyironFlow([wf]); pf.gui  for visual modePYTHON
ℹ PyironFlow GUI

Uncommenting pf.gui opens an interactive drag-and-drop workflow editor. It requires JupyterLab (not VSCode) and the correct anywidget version. For reliability, use wf.run() directly.

2

Optimization Process original BAM

Three-stage parameter identification strategy: a global search finds a good basin, then two local refinement stages converge to the optimum.

STAGE 1 — Global STAGE 2 — Local STAGE 3 — Refine DiffEvol Optimization Info-Opt-v3/ VisualizePlots best params NelderMead Optimization Info-Opt-v4/ VisualizePlots best params SingleStep Optimization Info-Opt-v5/ VisualizePlots IN718_ SDCHABOX .mat.best

Key configuration in Main.txt

MODEL_NAME  = SDCHABOX
MATERIAL_ID = IN718
SIMUL_PATH  = C:/path/to/simulop.exeMAIN.TXT

Key configuration in Input.py

# Set material and model — determines output filenames
material_id = 'IN718'
model_name  = 'SDCHABOX'

directories = Directories(
    parameterFile    = f'{material_id}_{model_name}.mat',
    bestParameterFile= f'{material_id}_{model_name}.mat.best',
    templateFile     = f'{model_name}.mat.tmpl',
    ... # optimizer stages, log files, etc.
)PYTHON
✅ Output

sim/IN718_SDCHABOX.mat.best — final calibrated parameter values used by all downstream KG steps.

Info-Opt-v5/Plots/Creep_550-*.png — simulation vs. experiment comparison plots.

3

Validation original BAM

Runs independent validation against held-out experimental data (not used during calibration) to confirm the calibrated parameters generalise.

Validation.txt (config) IN718_SDCHABOX.mat.best validation_exp/*.exp run_validation_workflow 5 steps: generate inputs → run sim → compare comparison_*.png (1 per test) IN718_SDCHABOX_validation_ report.txt RMSE / MAE metrics

Notebook cell

from validation_node import run_validation_workflow
import os; os.chdir("Validation/")

results = run_validation_workflow("Validation.txt", verbose=True)
# Returns dict with: num_tests, num_passed, report path, metricsPYTHON

Configure Validation/Validation.txt

MATERIAL_ID             = IN718
MODEL_NAME              = SDCHABOX
OPTIMIZED_PARAMS_FILE   = ../sim/IN718_SDCHABOX.mat.best
TEMPLATE_FILE           = ../sim/SDCHABOX.mat.tmpl
SIMUL_PATH              = C:/path/to/simulop.exeVALIDATION.TXT
4

KG Generation with CKAN Uploads new — IWM integration

A 7-node PyIron workflow that uploads all artefacts to CKAN first (to obtain stable URLs), then generates the knowledge-graph Excel template with URLs embedded, converts to Turtle RDF, and uploads the final TTL.

PHASE 1 — UPLOAD & GET URLS ZipExp ZipExperimentalData UploadZip UploadToCKAN → URL UploadMatCard IN718_SDCHABOX.mat.best URL UploadReport IN718_SDCHABOX_val_report.txt URL PHASE 2 — KG GENERATION & PUBLISH PopulateExcel reads Main.txt + Input.py + .mat.best 3 URLs ExcelToTTL RDF/Turtle Upload TTL CKAN ⚠ Manual step after upload dataportal.material-digital.de → Dataset → Fuseki tab → Create/Update → load uploaded TTL into triple store

Notebook cell

project_dir  = 'C:/path/to/Pyiron_Windows_06_10_2026'
MATERIAL_ID  = 'IN718'
MODEL_NAME   = 'SDCHABOX'

wf_kg = Workflow("kg_generation")

# STEP 1 — Zip experimental data
wf_kg.ZipExp = zipn.ZipExperimentalData(project_dir=project_dir)

# STEP 2 — Upload zip to CKAN → URL
wf_kg.UploadZip = ckan.UploadToCKAN(
    ttl_file    = wf_kg.ZipExp.outputs["str(zip_path)"],
    description = "Experimental creep test data for calibration"
)

# STEP 3 — Upload material card → URL
wf_kg.UploadMatCard = ckan.UploadToCKAN(
    ttl_file = f"{project_dir}/sim/{MATERIAL_ID}_{MODEL_NAME}.mat.best"
)

# STEP 4 — Upload validation report → URL
wf_kg.UploadReport = ckan.UploadToCKAN(
    ttl_file = f"{project_dir}/Validation/results/{MATERIAL_ID}_{MODEL_NAME}_validation_report.txt"
)

# STEP 5 — Generate Excel KG template with URLs embedded
wf_kg.Excel = aen.PopulateExcel(
    project_dir               = project_dir,
    exp_container_url         = wf_kg.UploadZip.outputs["resource_url"],
    material_card_url         = wf_kg.UploadMatCard.outputs["resource_url"],
    material_card_filename    = f'{MATERIAL_ID}_{MODEL_NAME}.mat.best',
    validation_report_url     = wf_kg.UploadReport.outputs["resource_url"],
    validation_report_filename= f'{MATERIAL_ID}_{MODEL_NAME}_validation_report.txt'
)

# STEP 6 — Convert Excel to Turtle RDF
wf_kg.TTL = ettn.ExcelToTTL(
    excel_file = wf_kg.Excel.outputs["str(output_excel)"]
)

# STEP 7 — Upload TTL to CKAN
wf_kg.UploadTTL = ckan.UploadToCKAN(
    ttl_file = wf_kg.TTL.outputs["str(ttl_path)"]
)

wf_kg.run()PYTHON
⚠ Manual Fuseki update required

Uploading the TTL to CKAN does not automatically update the Fuseki triple store. After wf_kg.run() completes:

  1. Go to dataportal.material-digital.de/dataset/bam-sdchabox-workflow-kg
  2. Click the Fuseki tab
  3. Click Create / Update → confirm to load the new TTL

Until this step is done, SPARQL queries in Sections 5–6 return stale data from the previous load.

✅ What gets created in CKAN

All resources land in dataset bam-sdchabox-workflow-kg at dataportal.material-digital.de · organization digitalmodelling.

5

Material Card Extraction via SPARQL new — IWM integration

Queries the Fuseki-backed knowledge graph to retrieve all calibrated SDCHABOX parameters for a material and exports them as a pivot-table CSV.

material_name "IN718" QueryMaterialCard SPARQL → CalibratedParameterSet Fuseki endpoint (POST) Authorization: raw JWT material_card_IN718.csv (pivot) DataFrame displayed inline

Notebook cell

MATERIAL_NAME = "IN718"

wf_sparql = Workflow("material_card_query")
wf_sparql.Query = sqn.QueryMaterialCard(
    material_name = MATERIAL_NAME,
    project_dir   = PROJECT_DIR,
)
wf_sparql.run()

# Output channel name = the return expression string
csv_path = wf_sparql.Query.outputs["str(output_csv)"].valuePYTHON

Example output (pivot table)

parameterunitcalibrated_valuemin_boundmax_boundtemperature_C
bk—1.2341.04.47550
bqMPa48200.0-200000200000550
sf—-9.31-15.0-3.0550
skMPa19.30.01500.0550
sn—4.53.010.0550
… all CalibratedParameterSet parameters …

SPARQL endpoint

https://dataportal.material-digital.de/dataset/bam-sdchabox-workflow-kg/fuseki/$/sparql

# Pattern: {ckan_base}/dataset/{dataset_slug}/fuseki/$/sparql
# Method:  POST  (Content-Type: application/x-www-form-urlencoded)
# Auth:    Authorization: <raw_jwt>   (no "Bearer " prefix)SPARQL ENDPOINT
6

Material Card Pass (MCP) new — IWM integration

Generates a self-contained, browser-ready HTML Material Card Pass by running 6 SPARQL queries against the Fuseki knowledge graph. The result is published to CKAN and the download URL is returned.

identifier "IN718" GenerateMaterialCardMCP ① Resolve card_id from KG ② Provenance (process, org, date) ③ Software & Equipment ④ Input & Control Parameters ⑤ Calibrated Parameters ⑥ Linked Files (inputs + outputs) ⑦ Material Information MCP_IN718_SDCHABOX.mat.best.html Self-contained HTML — 7 sections with inline CSS & navigation Published to CKAN resource_url returned as output channel dataset: bam-sdchabox-workflow-kg

Notebook cell

IDENTIFIER  = "IN718"  # material name OR full MaterialCard ID
PROJECT_DIR = 'C:/path/to/Pyiron_Windows_06_10_2026'

wf_mcp = Workflow("material_card_mcp")
wf_mcp.MCP = mcp.GenerateMaterialCardMCP(
    identifier  = IDENTIFIER,
    project_dir = PROJECT_DIR,
)
wf_mcp.run()

ckan_url = wf_mcp.MCP.outputs["str(resource_url)"].value
print(f"MCP published: {ckan_url}")PYTHON

MCP HTML sections

#SectionSPARQL sourceTypical rows
1ProvenanceProcess label, timestamp, organization1
2Software & EquipmentSoftwareArchitecture, InfrastructureEquipmentSet, ProcessSetup~8
3Input & Control ParametershasControlInfo + input DataSet parameters~59
4Calibrated ParametersCalibratedParameterSet (values, bounds, units)~16
5Linked FilesisInputFor + hasOutput (incl. MaterialCard, ValidationReport)~8
6Material InformationisRepresentativeFor → MaterialID1
ℹ Identifier resolution

The node accepts either a short material name ("IN718") or a full card ID ("IN718_SDCHABOX.mat.best"). It resolves to the MaterialCard identifier stored in the KG via two SPARQL fallback queries.

R

Dynamic Naming Convention

All output filenames are generated from MATERIAL_ID and MODEL_NAME. Changing the material or model only requires updating these values in Main.txt and Input.py — no hardcoded path edits elsewhere.

FilePatternExample
Calibrated parameters{MATERIAL_ID}_{MODEL_NAME}.mat.bestIN718_SDCHABOX.mat.best
Working parameter file{MATERIAL_ID}_{MODEL_NAME}.matIN718_SDCHABOX.mat
Validation report{MATERIAL_ID}_{MODEL_NAME}_validation_report.txtIN718_SDCHABOX_validation_report.txt
MCP HTMLMCP_{card_id}.htmlMCP_IN718_SDCHABOX.mat.best.html
SPARQL output CSVmaterial_card_{material_name}.csvmaterial_card_IN718.csv

Where to change identifiers

WhatFileKey
Material ID (e.g. IN718)Main.txtMATERIAL_ID = IN718
Model name (e.g. SDCHABOX)Main.txtMODEL_NAME = SDCHABOX
Optimizer file namesInput.pymaterial_id / model_name variables
Validation file namesValidation/Validation.txtMATERIAL_ID / MODEL_NAME
CKAN dataset slugckan_upload_nodes.pyNEW_CKAN_DATASET
CKAN API tokenckan_upload_nodes.pyNEW_CKAN_TOKEN
!

Troubleshooting

ValueError: ParseOutput can only parse callables with at most one return value

A helper function defined inside the @as_function_node body has its own return statement. PyIron's AST walker counts it. Fix: move all helpers to module level.

Widgets are empty / not rendering

JupyterLab is served from base (anywidget 0.9.x) while the kernel is pyiron_env (anywidget 0.11.x). Always launch JupyterLab from within pyiron_env. Workaround: use display(pf.gui) explicitly, or just call wf.run().

HTTP 404 on SPARQL endpoint

The endpoint only accepts POST. GET returns 404. Pattern: {ckan_base}/dataset/{slug}/fuseki/$/sparql. UUID-based URLs do not work.

SPARQL queries return 0 rows

The Fuseki triple store was not refreshed after the last TTL upload. Go to the dataset on CKAN → Fuseki tab → Create/Update to reload. Also verify the MaterialCard identifier in the KG matches your current MATERIAL_ID_MODEL_NAME.mat.best naming.

FileNotFoundError on mat.best or validation_report.txt

The optimizer or validation node has not been run yet with the new MATERIAL_ID_MODEL_NAME naming convention. Either run those steps first, or copy the old files: SDCHABOX.mat.best → IN718_SDCHABOX.mat.best.

Authorization: raw token — no "Bearer" prefix

The Fuseki endpoint and the CKAN API both use the same JWT token. Set it as Authorization: <raw_token> without the "Bearer " prefix.

pandas 3.0 — applymap removed

Use df.map(fn) instead of df.applymap(fn). This affects the MCP HTML table generation in mcp_dpp_node.py.