Am I doing the right thing? (regarding this little puppy)?

That puppy is way way way too young to be away from it's mother. Why would they take it away? Still it sounds like you are trying to do your best for it. It needs formula because it would still be getting milk from it's mother. Take it to the vet as soon as you can. They'll know how to properly care for it.