Calderdale Council is encouraging people to come forward with information that will help stop fly-tipping at Rawson Mill.
The Grade II listed building has been blighted by rubbish being dumped in the grounds, and has fallen into a state of disrepair.
Concerned local resident Nigel Paddon estimates the clean-up cost of the 11-acre site could be £90,000 per acre.
He said: “The building has experienced fly-tipping for years, and there has been only limited action to tidy it up - usually the visible bits - hence, the mess is horrendous.
“Police have been called on several occasions to deal with trespassing, and there are signs of vandalism and graffiti.
“It is currently up for sale pending renovations, but it won’t be worth buying if the abuses don’t cease.
Coun Susan Press, Calderdale Council’s Cabinet Member for Communities and Neighbourhood Services, said: “Fly-tipping is illegal and blights our communities. We all want clean streets, so we won’t hesitate to take action against those who break the law by dumping rubbish.
“We are aware of a number of fly-tipping incidents around Rawson Mill and the adjacent Old Lane, and have made several prosecutions in the past. We have used portable cameras in the area to try to combat the issue.
“The land is privately owned, meaning it’s the landowner’s responsibility to prevent and remove fly-tipping.
“We have contacted the owner and supported them to secure the site. However, the vast size of the area and the numerous access points limit the effectiveness of preventative measures, so we urge people to come forward with evidence to help us catch the culprits – that could include information like names, dates, times and vehicle registration numbers.”