+ Start a Discussion

Prevent / block datamining ?

Hi all.


We're setting up a large custom authenticated website in visual force for our customers. Exploring the visitation statistics of our current website we've noticed there are some people datamining our pages. 


We'd like to implement something into our Visual force pages that can detect and block datamining/page scraping. Has anyone faced this already, or know of a good solution ?  AppExchange nor google didn't give any results.





Santhosh KumarSanthosh Kumar

You say, it is authenticated VF pages. Then won't you know the users who are doing the data-mining and take necessary actions (penalizing or inactiviating those users)?


If that still doesn't answer your question, you can add Captcha to your pages so that only human users can do something. Note that Captcha for authenticated users is really an inconvenience. So you have balance the risk of inconvenience and risk of your content data-scrapped.


Actually, how to know what users do which or how many request ? Maybe there are logging features that I'm unaware of, is there more than an overview of the bandwidth & request time for our site ?


If we would have to implement our own logging mechanism, we'd need to write a custom object record on every page request, which would be a LOT of tiny records. I feel this isn't really the best use of SF's infrastructure.