David Freeman
Friday, February 07, 2019 at 11:00 AM in 380 Soda Hall
Title: The Abuse Uncertainty Principle, and Other Lessons Learned from Fighting Abuse on the Internet // Adversarial Machine Learning in Real Life: Examples, Lessons, and Challenges
Abstract: Fighting spam, phishing, and other forms of abuse on the internet is typically seen as a detection problem: find signals that will identify the bad guys and then use these signals to block them. In this talk, I argue that the most difficult part of fighting abuse is not detecting and blocking the bad guys—it's figuring out whether they're there in the first place. What's the "background level" of spam and fake accounts? How can we figure out what our detection systems are missing? Which abuse problem is the most important one to work on right now? In this talk, I will show how good measurement of abuse unlocks both prioritization of work and analysis of impact. I will present several approaches that Facebook's integrity teams have used to measure and prioritize their problems, including user reports, human labeling, and automated labeling, and offer scenarios in which each of these should and shouldn't be used. I will also introduce the "Abuse Uncertainty Principle" which says that you can use a metric for measurement or detection, but not both. The Uncertainty Principle implies that measurement is never a finished project, but I will offer strategies for ensuring that your metrics are good enough to inform key decisions. Armed with these tools, you can go back to your product and find out how much abuse it's attracting, how good you are at stopping it, and where you need to invest next. At Facebook we have numerous machine learning systems intended to protect our platform from various forms of abuse, and we see millions of real-life adversarial examples every day as bad actors try to get around our systems to perpetrate their misdeeds. In this talk I will give some examples of attacks we have seen, approaches we've taken to mitigate them, and challenges still facing us.
Bio: David Freeman is a research scientist/engineer at Facebook working on integrity problems, with a particular focus on fake engagement, scraping, and automation detection. He previously led anti-abuse engineering and data science teams at LinkedIn. He is an author, presenter, and organizer at international conferences on machine learning and security, such as Enigma, NDSS, WWW, and AISec, and has written (with Clarence Chio) a book on Machine Learning and Security published by O'Reilly. He holds a Ph.D. in mathematics from UC Berkeley and did postdoctoral research in cryptography and security at CWI and Stanford University.