If you include a sample of someone else's music in your own work, then you need to get permission to use it. There are two different permissions in order to legally use a sample from an existing recording in your own music:
A license for the usage of the master recording (Recording copyright). This is often owned by the label who released the track.
A license for the usage of the underlying composition (Song copyright) This belongs to the songwriter or the publisher.
In order to use a sample legally, you need to have clearance from the owners of both copyrights.
The cost of a sample will vary massively, depending on the scale of the release, how popular the original track is and how well established an artist you are. Some record companies will want a flat-fee, known as a 'buy-out'. Others will negotiate a percentage of the profits, though they may also insist on an advance. The longer the sample, the more you may have to shell out.
Re-recording a sample
Unfortunately, on the publishing rights side of things, if someone says 'no' and refuses consent it's pretty much 'game over' for your sample clearance. Legally speaking you won't be able to proceed even if the people who have the recording rights have said 'yes'. However, if you get a 'yes' from the publishing rights owner but a 'no' from the recording rights owner (or they're asking for too much money), there is one thing you can do. You can re-create the sample by playing it yourself or getting someone else to perform it for you (there are companies who specialise in this). That way, you're not using the actual recording and so you're not infringing the record company's copyright. This doesn't get you out of the need to do a deal with the publishers, but it frees you from needing the record company's permission. Publishing companies will also want a royalty but not normally an advance.
Recording covers
If you want to record a cover of someone else’s song you will need to get permission from the copyright owner, which will either be the person who wrote the song or the person/company who the rights were assigned to. The easiest way to find out who owns the copyright is to search on databases like TEOSTO, BMI, ASCAP, SESAC or look on the PRS For Music website. You need to get in touch with owner of the copyright to let them know when you plan to release the song, how many copies of the recording you will make, the name of your band and any other details about your plans. How much you'll have to pay will vary. You may have to pay a one-off fee or a percentage of any sales.