...
-x SHARP_COLL_JOB_QUOTA_PAYLOAD_PER_OST=256
Note: It is recommended to play around with the parameters and test it compare to the non-SHARP allreduce tests.
Debugging SHARPÂ
1. Add log level 3 (information) to get the log for communicator creation.
-x SHARP_COLL_LOG_LEVEL=3