Usually an invention is credited to the person who conceives the idea, works it out and proves it. The device need not be built. Atom bomb idea and theory was worked out and release of tremendous atomic energy through a nuclear chained reaction through fission process was patented by a Scientist called "Leo Szilard". He patented the idea of nuclear reactor in 1936. So credit goes to him for invention.
He wrote a letter to Einstein and Einstein wrote to the president of USA regarding atomic bomb related activity. UsA started a project called Manhattan project that built the atomic bomb during the world war II. The scientists involved in that project are many, with J. Robert Oppenheimer being in charge of it. Scientists are many: Neils Bohr, Emilio Segre, Leo Szilard, Felix Bloch, Enric Fermi, Edward Teller and some others. Oppenheimer is often regarded as the father of atomic bomb.