Internal:DQHED

Distributed Quantum Hadamard Edge Detection (DQHED)

Literature Review

This report was generated by reviewing publicly available research as of March 2022.

Weekly Reports

Weeks of 2Mar and 12Mar

Generated the following notes on current methods in quantum edge detection.

Week of 19Mar

In an effort to understand it - reviewed, commented, and added typing to Emmanuel's QHED code in the quantum repository at wahab.hpc.odu.edu:/RC/group/RC_NIKOS/CodeRepository/quantum.git

Weeks of 26Mar and 2Apr

Created a separate script for Multi-node Quantum Hadamard Edge Detection (QHED) with a workflow for parallel quantum circuit simulation and execution. Current progress is stored on the qhed branch. The three main tasks eligible for concurrency are:

1. Circuit description

2. Transpilation

3. Execution

In addition, the requirement for padding was removed, as it is not necessary. Only one pixel per run is wasted to provide overlap between subimages.

Week of 9Apr

Researched the SLURMCluster module from Dask with respect to the MQHED problem. All that is required is to provide the module a description of the nodes. The Scheduler will create and submit a job script to SLURM, which will function to automatically request nodes. Once this is complete, Dask Futures can be used to provide arbitrary Python methods with arguments for the SLURMCluster to manage. Relevant methods are dask.distributed.Client.map and dask.distributed.[result, gather, as_completed]. Sample usage will be as follows, with the SLURMCluster parameters specified by a .yaml file:

   from dask.distributed import Client, as_completed
   from dask.jobqueue import SLURMCluster

   cluster = SLURMCluster()
   cluser.adapt(minimum_jobs=min, maximum_jobs=max)
   client = Client(cluster)
   .
   .
   .
   results = []
   futures = client.map(func, args)
   results = client.gather(futures)    

The quantum portion of the algorithm has been implemented and run on an 11th Gen Intel Core i7-11800H. It remains to implement the code process the results of the circuit runs and convert them into an edge detected image.

Week of 16Apr

The code example from the 9Apr section fails to allocate any cores or memory on Wahab, for reasons unknown. An alternative method of using a batch script with srun and possibly mpirun to execute a Python script which calls dask_mpi.initialize() and Client() has been successful on a sample script, but it is extremely slow and essentially infeasible for large numbers of mpi slots. An example is provided below

   from dask.distributed import Client, progress
   import dask_mpi

   dask_mpi.initialize(interface='ib0') # Optional parameter to specify InifiniBand
   client = Client()
   .
   .
   .
   futures = client.map(func, args)
   progress(futures)

HPC ITS has been contacted and have suggested that using dask_mpi with srun, but not mpirun, is the best course of action. However, this method is currently much slower than running dask on a local machine with the LocalCluster provided to the Client object. It has demonstrated itself to be infeasible in the current configuration when running a simple program with more than 4 allocated tasks.

Further experimentation in using dask with the Wahab SLURM scheduler is needed to determine its efficacy. It may be advisable to get in contact with dask contributors to help implement dask on ODU CS HPC clusters, as it would benefit both our group and the school. If it deemed infeasible, other methods or software may need to be investigated.

References

Emmanuel's Contibutions

Proposal Additions for QHED December 2022[1]

Software

Qiskit

Dask