Betting firm William Hill has been hit with a £6.2m penalty package for breaching anti-money-laundering and social responsibility regulations.
The Gambling Commission said the company did not do enough to ensure prevention measures were effective.
As a result, 10 customers were able to deposit money linked to criminal offences and William Hill gained £1.2m.
The company was found to have not done enough to determine the source of the money or if they were problem gamblers.
The penalty is the second biggest imposed by the Commission, following last year's £7.8m action against betting firm 888.
In a statement, the Gambling Commission said William Hill's senior management "failed to mitigate risks and have sufficient numbers of staff to ensure their anti-money-laundering and social responsibility processes were effective".
William Hill said it had co-operated with the Commission during the investigation. It added it had committed to an independent review and would work to implement any recommendations following this review.
Gambling Commission executive director Tim Miller told the BBC that William Hill should have been "checking the source of money and understanding their customers and ensuring that potentially vulnerable customers are properly protected".
The investigation by the Gambling Commission covered the period between November 2014 and August 2016.
It gave some examples of where it said William Hill's controls were insufficient:
- One customer deposited about £541,000 over a 14-month period after William Hill assumed his potential income was £365,000 a year, "based on a verbal conversation and without further probing". In fact, the customer was earning about £30,000 a year and was funding his gambling habit by stealing from his employer.
- Another customer exceeded deposits of £147,000 over 18 months with an escalating spend and losses of £112,000. William Hill's systems identified the issue, "but its only response over a 12-month period was to send two automated social responsibility emails. Our view is that this action alone was not sufficient, given the customer's gambling behaviour, coupled with the severity of the losses."
- A customer was allowed to deposit £653,000 in an 18-month period which activated a financial alert at William Hill. The alert resulted in a grading of "amber risk", which meant the customer's profile should have been reviewed under the company's anti-money-laundering policy. The file was marked as passed to managers for review, but that did not happen because of a systems failure. The customer was able to continue gambling for a further six months, despite continuing to activate financial alerts.
William Hill will pay more than £5m for breaching the regulations and "divest themselves of the £1.2m they earned from transactions with the 10 customers", the Commission said.
The company now has to appoint external auditors to review the effectiveness and implementation of its anti-money laundering and social responsibility policies and procedures
Neil McArthur, executive director at the Gambling Commission, said: "This was a systemic failing at William Hill which went on for nearly two years and today's penalty package - which could exceed £6.2m - reflects the seriousness of the breaches."
Analysis: Dominic O'Connell, Today business presenter
For the wider industry, the timing of William Hill's fine could not be worse.
The government is weeks away from a crucial decision on fixed-odds betting terminals, the gambling machines found in high-street bookmakers up and down the country. The maximum stake could be cut from £100 to as low as £2.
If it is at the bottom end of the range, the consequences for industry profitability would be enormous.
There would also be far-reaching ramifications for the High Street - some estimate as many as 3,000 shops would close - and for revenue for the horse-racing industry.
'Get this right'
In a statement, William Hill chief executive Philip Bowcock said: "William Hill has fully co-operated with the Commission throughout this process, introducing new and improved policies and increased levels of resourcing.
"We have also committed to an independent process review and will work to implement any recommendations that emerge from that review.
"We are fully committed to operating a sustainable business that properly identifies risk and better protects customers.
"We will continue to assist the Commission and work with other operators to improve practices in the areas identified."
Mr Miller said the Gambling Commission was sending a message to the industry as a whole that "you need to take your responsibility seriously, you need to get this right".
"We are always taking regulatory action and I don't think this will be the last action we take," he added.