Spectrum Computing

Spectrum Computing

Connect with Spectrum Computing subject matter experts and discuss how hybrid cloud Solutions from IBM meet today's business needs.

 View Only

Orchestrating Hybrid Quantum–Classical Workflows with IBM LSF: Inside the SQD Workflow Demo at SC25

By Gábor Samu posted 01/06/26 04:15 PM

  

As we enter 2026, it seems that SC25 is far off in our rearview mirror. But it's only been a bit over a month since the HPC world converged on St. Louis, Missouri for the annual Supercomputing 2025 (SC25) event. SC25 signaled one emerging trend: the exploration of hybrid workflows combining quantum and classical computing, offering a look at how these technologies can work synergistically over time. This was indeed the main topic of the 1st Annual Workshop on Large-Scale Quantum-Classical Computing, a workshop which I found to be very insightful. 

At the IBM booth, we showcased how IBM LSF can schedule and orchestrate a hybrid quantum–classical workflow across IBM Quantum systems and classical x86 compute.  The demo featured the Sample-based Quantum Diagonalization (SQD) workflow, to estimate the ground-state energy of a Hamiltonian representing a molecular system. SQD is part of the IBM Qiskit add-ons. 

Before diving into the details on what was demonstrated at SC25, and how LSF was used to manage the workflow, I would like to acknowledge that this work was supported by the Hartree Center for Digital Innovation, a collaboration between UKRI-STFC and IBM. The demonstration was created in close collaboration with Vadim Elisseev and Ritesh Krishna from IBM Research, alongside Gábor Samu and Michael Spriggs from IBM. Additionally, this post does not aim to provide an in-depth look at SQD itself. Rather the focus is on how LSF can manage hybrid quantum-classical workflows across a heterogeneous environment comprised of both quantum and classical resources. 

Hybrid workflows are not new

For three decades, we have seen the use of accelerators in HPC to drive performance—from GPUs to FPGAs and other specialized architectures. Effective scheduling of tasks in these heterogeneous environments has always been a key consideration for efficiency, scalability—and to maximize the ROI in commercial HPC environments. As resource topologies grow more complex, scheduling must account for characteristics such as connectivity, latency, and dependency constraints across increasingly diverse infrastructures. Quantum Processors (QPUs) are now making their appearance as complementary resources within HPC workflows, aim at challenges such as specific optimization problems, many-body physics and quantum chemistry. 

Demo details

The IBM LSF cluster was deployed on IBM Cloud using the LSF Deployable Architecture, which rapidly deploys and configures a ready-to-use HPC environment. IBM Research provided integration components for LSF in the form of esub and jobstarter scripts. These scripts enable LSF to query the cloud-based IBM Quantum Platform to determine which QPUs are available for a given user account and meet the qubit requirements specified at job submission. The list of eligible QPUs is then sorted by queue length, and the system with the shortest queue is selected as the target for the quantum circuit. These integration scripts (esub and jobstarter) are intended to be made open source at a later time. 

The LSF environment was deployed on IBM Cloud using the LSF Deployable Architecture v3.1.0 and is configured as follows: 

  • LSF 10.1.0.15
  • RHEL 8.10
  • IBM Cloud profile bx2-16x64 (compute hosts)

The IBM Qiskit package versions used:

  • qiskit v2.2.1
  • qiskit-addon-sqd v0.12.0
  • qiskit-ibm-runtime v0.43.0

The SQD Python program is available as part of the IBM Qiskit Add-ons (see details here). For this demonstration, the original monolithic SQD script was refactored into four smaller Python programs—each representing a distinct step in the workflow. These steps map directly to LSF jobs, enabling orchestration of the workflow across the quantum and classical HPC resources as shown in the architecture diagram (Figure 1):

  • Stage 1 map the inputs to a quantum problem.  
  • Stage 2 optimizes the problem for quantum hardware execution—this is where the circuit is transpiled and optimized for the target QPU
  • Stage 3 executes the circuit on the QPU using Qiskit primitives
  • Stage 4 performs post-processing and returns the result in the desired classical format
Figure 1 LSF hybrid quantum-classical demo (Vadim Elisseev, IBM Research)
Figure 1 LSF hybrid quantum-classical workflow demo (Vadim Elisseev, IBM Research)

For this demonstration, we used IBM LSF Application Center—a web-based interface for job submission and management. LSF Application Center supports application templates, which simplify job submission by providing predefined forms. Templates were created for both the SQD workflow and the Jupyter Notebook application, which is used to visualize the workflow results.

Demo execution steps

  • We start by using the SQD template to submit an instance of the SQD workflow (Figure 2) which is used to calculate an approximate ground-energy state of the nitrogen molecule (N2). The submission form is customized to let users specify the script for each step of the workflow and specify the desired number of qubits required on the QPU for the quantum circuit. This parameter is used by LSF to select the appropriate quantum system from the available resources. Note that jobs are submitted to LSF with a done dependency condition, ensuring that each stage runs only after the previous one completes successfully. Stage 2 begins after Stage 1, Stage 3 follows Stage 2, and Stage 4 executes once Stage 3 has finished

Figure 2 SQD submission form

Figure 2 SQD submission form

  • Next, we submit an instance of the Jupyter Notebook to monitor the workflow initiated in Step 1. This notebook is designed for this demonstration to visualize the status of each workflow step, displaying results as they successfully complete. Figure 3 shows the Jupyter submission form. 
Figure 3 Jupyter Notebook submission form

Figure 3 Jupyter Notebook submission form

  • The Workload view in the LSF Application Center can be used to monitor the progress of each job within the workflow. Additionally, the Jupyter Notebook instance can be accessed here via the provided hyperlink. Figure 4 shows the workload view in LSF Application Center. This shows a list of jobs in the LSF system. 
Figure 4 LSF Application Center workload view
Figure 4 LSF Application Center workload view
  • As each stage of the SQD workflow completes, the Jupyter Notebook displays the corresponding output in new browser tabs. This includes qubit coupling maps for the QPUs available on the IBM Quantum Platform for the specific account, a diagram of the circuit mapped to the selected QPU, readings from the QPU, and a plot of the estimated ground-state energy of the N2 molecule. 
Figure 5 Output from each step of the SQD workflow (Vadim Elisseev, IBM Research)
Figure 5 Output from each step of the SQD workflow (Vadim Elisseev, IBM Research)
  • Given that demo environment was built using the LSF Deployable Architecture, IBM Cloud Monitoring is automatically configured. It provides a dashboard for the underlying cloud infrastructure, including detailed hardware metrics. In addition, an LSF Dashboard is available through IBM Cloud Monitoring, showing overall cluster metrics such as total jobs, job status, and queue distribution, along with scheduler performance trends over time. IBM Cloud Monitoring infrastructure view and LSF dashboard are shown in Figure 5. 
Figure 6 IBM Cloud Monitoring: Infrastructure view, and LSF dashboard
Figure 6 IBM Cloud Monitoring: Infrastructure view, and LSF dashboard
A video recording of the end-to-end demonstration can be found here. 

Conclusions

This demo marked a milestone by demonstrating that IBM Spectrum LSF can seamlessly orchestrate quantum and classical compute resources for a unified workflow. This example demonstrates a practical approach to integrating quantum capabilities into an existing HPC environment running IBM LSF. 

This capability lays the foundation for hybrid computing pipelines that integrate emerging quantum hardware into established HPC environments. As organizations adopt these architectures and tools mature, we can expect production-grade workflows tackling complex problems across domains. The future of HPC is not a choice between classical or quantum—it is their convergence, working together to unlock new computational possibilities. 

The topic of scheduling for hybrid quantum-classical environments will be the subject of an upcoming paper "On Topological Aspects of Workflows Scheduling on Hybrid Quantum - High Performance Computing Systems" by Vadim Elisseev, Ritesh Krishna, Vasileios Kalantzis, M. Emre Sahin and Gábor Samu. 

1 comment
53 views

Permalink

https://community.ibm.com/community/user/blogs/gbor-samu/2026/01/06/orchestrating-hybrid-quantum-classical-workflows

Comments

Suman Suhag
01/08/26 09:29 AM

Wow, 

It is great.

LSF 10.1.15 seamlessly orchestrates Qiskit Runtime Primitives across IBM Quantum Platform QPUs + bx2-16x64 x86 HPC through pure bsub -w "prev.done" dependency chains + dynamic QPU discovery via extensible esub/jobstarter hooks. This productionizes hybrid quantum-classical workflows without Qiskit modifications or custom schedulers.

























SQD N2 ground-state workflow → 4-stage LSF DAG:
┌─ Stage1: x86 → Hamiltonian→Qubitization [LSF job]
├─ Stage2: x86 → Qiskit Transpiler → target QPU [LSF job]  
├─ Stage3: QPU → Sampler Primitive → bitstrings [LSF external]
└─ Stage4: x86 → post-process → |ψ₀⟩ energy [LSF job]

bsub -w "done(Stage1)" -J Stage2 sqd-transpile.py
bsub -w "done(Stage2)" -J Stage3QPU esub_qpu_run.sh  # QPU job
bsub -w "done(Stage3QPU)" -J Stage4 sqd-postprocess.py

QPU 

























bash




# esub queries IBM Quantum API → dynamic resource selection
curl -H "api-token: $QTOKEN" \
  "https://quantum.cloud.ibm.com/api/QuantumSystems/v1/groups/real/devices"

→ Filter: qubits>=27 (N2 Hamiltonian)
→ Sort: queue_length ASC
→ Select: heron-c1 (shortest queue, 127q)

LSF extensibility unlocks quantum





















LSF Application Center → Template-driven citizen science:
├── SQD Workflow Template:
│   ├── qubit_count=27 → auto-QPU selection
│   ├── stage_scripts[] → 4 Python microservices
│   └── dependency: "done" auto-DAG
├── Jupyter Monitor Template → real-time job visualization
└── bx2-16x64 auto-scaling → classical HPC backbone

Production metrics decoded:

























text




Qiskit Runtime Primitives → 10x latency reduction vs raw circuits
LSF esub/jobstarter → 30min integration (open source imminent)
bx2-16x64 IBM Cloud → 16x64c/2TB per node, 100G interconnect
IBM Cloud Monitoring → unified classical/quantum dashboards




























LSF Resource Classes evolve:
1992: CPUs → resource="x86"
2005: GPUs → resource="tesla-v100"  
2025: QPUs → resource="heron-127q-queue<5"

bsub -R "rusage[qpulist=heron]" sqd-stage3.py



























1. LSF Deployable Architecture → bx2-16x64 IBM Cloud (90min)
2. pip install qiskit-addon-sqd==0.12.0 qiskit-ibm-runtime==0.43.0
3. wget esub_qpu_discovery.sh jobstarter_qpu_run.sh (open source)
4. LSF Application Center → SQD template → bsub N2-workflow
5. Jupyter monitor → real-time QPU results + energy convergence plot

SC25 tipping point: QPUs graduate from research toys to LSF resource classes. Nitrogen molecule ground-state (27q) → portfolio optimization (100q) → drug discovery (1000q).

 LSF schedulers ate quantum like they ate every accelerator decade after decade.

LSF customer inherits production quantum workflow orchestration. No new licenses. No retraining. Quantum joins GPUs in LSF's resource taxonomy. Deploy the SC25 reference architecture. Production hybrid workflows start today.