In 1824, John Cadbury began selling tea, coffee, and drinking chocolate, which he produced himself, at Bull Street in Birmingham, England. He later moved into the production of a variety of cocoa and drinking chocolates, made in a factory in Bridge Street and sold mainly to the wealthy because of the high cost of production. John Cadbury became a partner with his brother Benjamin and the company they formed was called ‘Cadbury Brothers of Birmingham’.
The brothers opened an office in London and in 1854 they received the Royal Warrant as manufacturers of chocolate and cocoa to Queen Victoria. In the 1850s the industry received a much needed boost, with the reduction in the high import taxes on cocoa, allowing chocolate to be more affordable to everybody.
Info gleaned from Wikipedia