Advice needed for small project on deep reinforcement Learning

I'm a student and want to create my own RL project to add on CV and give myself some experience. I have implemented Relational Deep Reinforcement Learning framework tested on boxworld (See DeepMind "Relational Deep Reinforcement Learning" paper (2018)). The paper uses an actor-critic method but this is a single player. What I want to do now is add one more agent and that the reward can be shared and see if the agent would learn faster. I also have the said "shared-reward actor-critic" framework in python but this is from another studies. Due to my lack of experience, I'm stuck on how to create a multi-agent environment with this implementation of this share-reward framework into the current work. I want some guidance by experts that can teach me how to do so.