Internal:DQHED

Distributed Quantum Hadamard Edge Detection (DQHED)

Literature Review

This report was generated by reviewing publicly available research as of March 2022.

Weekly Reports

Weeks of 2Mar and 12Mar

Generated the following notes on current methods in quantum edge detection.

Week of 19Mar

In an effort to understand it - reviewed, commented, and added typing to Emmanuel's QHED code in the quantum repository at wahab.hpc.odu.edu:/RC/group/RC_NIKOS/CodeRepository/quantum.git

Weeks of 26Mar and 2Apr

Created a separate script for Multi-node Quantum Hadamard Edge Detection (QHED) with a workflow for parallel quantum circuit simulation and execution. Current progress is stored on the qhed branch. The three main tasks eligible for concurrency are:

1. Circuit description

2. Transpilation

3. Execution

In addition, the requirement for padding was removed, as it is not necessary. Only one pixel per run is wasted to provide overlap between subimages.

Week of 9Apr

Researched the SLURMCluster module from Dask with respect to the MQHED problem. All that is required is to provide the module a description of the nodes. The Scheduler will create and submit a job script to SLURM, which will function to automatically request nodes. Once this is complete, Dask Futures can be used to provide arbitrary Python methods with arguments for the SLURMCluster to manage. Relevant methods are dask.distributed.Client.map and dask.distributed.[result, gather, as_completed]. Sample usage will be as follows, with the SLURMCluster parameters specified by a .yaml file:

   from dask.distributed import Client, as_completed
   from dask.jobqueue import SLURMCluster

   cluster = SLURMCluster()
   cluser.adapt(minimum_jobs=min, maximum_jobs=max)
   client = Client(cluster)
   .
   .
   .
   results = []
   futures = client.map(func, args)
   results = client.gather(futures)    

The quantum portion of the algorithm has been implemented and run on an 11th Gen Intel Core i7-11800H (16 physical cores, 32 logical cores). It remains to implement the code process the results of the circuit runs and convert them into an edge detected image.

Week of 16Apr

The code example from the 9Apr section fails to allocate any cores or memory on Wahab, for reasons unknown. An alternative method of using a batch script with srun and possibly mpirun to execute a Python script which calls dask_mpi.initialize() and Client() has been successful on a sample script, but it is extremely slow and essentially infeasible for large numbers of mpi slots. An example is provided below

   from dask.distributed import Client, progress
   import dask_mpi

   dask_mpi.initialize(interface='ib0') # Optional parameter to specify InifiniBand
   client = Client()
   .
   .
   .
   futures = client.map(func, args)
   progress(futures)

HPC ITS has been contacted and have suggested that using dask_mpi with srun, but not mpirun, is the best course of action. However, this method is currently much slower than running dask on a local machine with the LocalCluster provided to the Client object. It has demonstrated itself to be infeasible in the current configuration when running a simple program with more than 4 allocated tasks.

Further experimentation in using dask with the Wahab SLURM scheduler is needed to determine its efficacy. It may be advisable to get in contact with dask contributors to help implement dask on ODU CS HPC clusters, as it would benefit both our group and the school. If it deemed infeasible, other methods or software may need to be investigated.

IT has corrected some of the errors, but the dask_mpi.initialize() method still seems to take a prohibitive amount of time to set up communication between processors/nodes. Further investigation is required.

Week of 23Apr

A fix was found for the dask-jobqueue cluster module. The issue was the module was not prepending the required environment flags to the batch scripts it generated. A workaround was found by using the `env_extra` parameter in the SLURMCluster constructor to prepend module loading and the `crun` command to the script. There are a number of other useful options in Dask which might be helpful in the future.

With this fix, a text script was successfully run which allocated the correct number of jobs across the correct number of nodes. It now remains to refactor the DQHED script for the multi-node environment, as my new understanding of the IPC subtleties makes me want to refactor a few parts of the existing script.

References

Emmanuel's Contibutions

Proposal Additions for QHED December 2022[1]

Software

Qiskit

Dask