View source for Internal:DQHED

'''Distributed Quantum Hadamard Edge Detection (DQHED)'''

= Literature Review =

Sections 1 and 2 of this [https://www.overleaf.com/read/jxnvpyysnkpx report] was generated by reviewing publicly available research as of March 2022.

= Methods =

The methods used can be found in Section 3.1 of this [https://www.overleaf.com/read/jxnvpyysnkpx report].

= Weekly Reports =

=== Weeks of 2Mar and 12Mar ===

Generated the following [https://www.overleaf.com/read/jxnvpyysnkpx notes] on current methods in quantum edge detection.

=== Week of 19Mar ===

In an effort to understand it - reviewed, commented, and added typing to Emmanuel's QHED code in the quantum repository at wahab.hpc.odu.edu:/RC/group/RC_NIKOS/CodeRepository/quantum.git

=== Weeks of 26Mar and 2Apr ===

Created a separate script for Multi-node Quantum Hadamard Edge Detection (QHED) with a workflow for parallel quantum circuit simulation and execution. Current progress is stored on the qhed branch. The three main tasks eligible for concurrency are:

1. Circuit description

2. Transpilation

3. Execution

In addition, the requirement for padding was removed, as it is not necessary. Only one pixel per run is wasted to provide overlap between subimages.

=== Week of 9Apr ===

Researched the [https://jobqueue.dask.org/en/latest/generated/dask_jobqueue.SLURMCluster.html SLURMCluster] module from Dask with respect to the MQHED problem. All that is required is to provide the module a description of the nodes. The Scheduler will create and submit a job script to SLURM, which will function to automatically request nodes. Once this is complete, Dask [https://docs.dask.org/en/stable/futures.html Futures] can be used to provide arbitrary Python methods with arguments for the SLURMCluster to manage. Relevant methods are dask.distributed.Client.map and dask.distributed.[result, gather, as_completed]. Sample usage will be as follows, with the SLURMCluster parameters specified by a .yaml file:

    from dask.distributed import Client, as_completed
    from dask.jobqueue import SLURMCluster

    cluster = SLURMCluster()
    cluser.adapt(minimum_jobs=min, maximum_jobs=max)
    client = Client(cluster)
    .
    .
    .
    results = []
    futures = client.map(func, args)
    results = client.gather(futures)    

The quantum portion of the algorithm has been implemented and run on an 11th Gen Intel Core i7-11800H (16 physical cores, 32 logical cores). It remains to implement the code process the results of the circuit runs and convert them into an edge detected image.

=== Week of 16Apr ===

The code example from the 9Apr section fails to allocate any cores or memory on Wahab, for reasons unknown. An alternative method of using a batch script with srun and possibly mpirun to execute a Python script which calls dask_mpi.initialize() and Client() has been successful on a sample script, but it is extremely slow and essentially infeasible for large numbers of mpi slots. An example is provided below

    from dask.distributed import Client, progress
    import dask_mpi

    dask_mpi.initialize(interface='ib0') # Optional parameter to specify InifiniBand
    client = Client()
    .
    .
    .
    futures = client.map(func, args)
    progress(futures)

HPC ITS has been contacted and have suggested that using dask_mpi with srun, but not mpirun, is the best course of action. However, this method is currently much slower than running dask on a local machine with the LocalCluster provided to the Client object. It has demonstrated itself to be infeasible in the current configuration when running a simple program with more than 4 allocated tasks.

Further experimentation in using dask with the Wahab SLURM scheduler is needed to determine its efficacy. It may be advisable to get in contact with dask contributors to help implement dask on ODU CS HPC clusters, as it would benefit both our group and the school. If it deemed infeasible, other methods or software may need to be investigated.

IT has corrected some of the errors, but the dask_mpi.initialize() method still seems to take a prohibitive amount of time to set up communication between processors/nodes. Further investigation is required.

=== Week of 23Apr ===

A fix was found for the dask-jobqueue cluster module. The issue was the module was not prepending the required environment flags to the batch scripts it generated. A workaround was found by using the `env_extra` parameter in the [https://jobqueue.dask.org/en/latest/generated/dask_jobqueue.SLURMCluster.html SLURMCluster] constructor to prepend module loading and the `crun` command to the script. There are a number of other useful options in Dask which might be helpful in the future. It should also be noted that dask-jobqueue does not just create a batch script. It creates a batch script which forces the creation of a dask.worker with the following python module command:

    python -m distributed.cli.dask_worker

With this fix, a text script was successfully run which allocated the correct number of jobs across the correct number of nodes. It now remains to refactor the DQHED script for the multi-node environment, as my new understanding of the IPC subtleties makes me want to refactor a few parts of the existing script.

The current, unoptimized code was run on a 64 x 64 image and 256 x 256 image using 5 qubits. The results can be seen in the table below. Strange errors were encountered when trying to run the 256 x 256 image with more cores, so that will need to be addressed.

[[File:Initial_mqhed.PNG|300px|center]]

=== Week of 30Apr ===

'''Subimage Extraction'''

Finished results extraction and image reconstruction implementations. This, in turn, revealed that the current quantum circuit implementation is just a image encoder, not edge detection. The circuit must be modified to encode redundant pixels and then add a Hadamard gate.

'''Image Reconstruction'''

Qiskit returns a dictionary of circuit results, of which the keys are the strings representing binary numbers (i.e. "001001"). This must be converted back into an array representing the subimage from which the quantum circuit was built. Each even index is removed, leaving the relevant differences.

After extraction, all subimages must be aggregated to reconstruct the edge-detected image. This is difficult for Python, since the GIL prevents the use of shared memory. Thus, each subimage must be saved to the final data structure sequentially.

= References =

==== Emmanuel's Contibutions ====
Proposal Additions for QHED December 2022[https://olddominion-my.sharepoint.com/:b:/r/personal/nchrisoc_odu_edu/Documents/Spring_23/quantum_edge_detection/proposal_additions.pdf?csf=1&web=1&e=u6iEeh]

==== Software ====

[https://qiskit.org/ Qiskit]

[https://docs.dask.org/en/stable/ Dask]