With data sets gathered since 2011, Black Book rankings are based on client experience scores obtained from the 800,000 crowdsourced ballots cast and available through mobile apps, web surveys, remote polling instruments, interview calls and on-site trade shows and user groups throughout the year. They represent the opinions of healthcare professionals and clinicians from more than 4,600 hospitals, 600 IDNs, 2,800 clinics, 109,000 physician practices, 250 health plans, and nearly 500,000 healthcare industry consumers, and account for 8,000 products and services from more than 2,900 vendors. Black Book Research stops bots from stuffing the ballot box for the 2023 client rankings from behind the scenes, where the company employs a third-party cybersecurity watchdog software that protects polling apps and websites from hacks and bots in real-time. Using a combination of machine learning, behavior, and device intelligence, as well as dozens of data points, it rapidly determines whether a user is genuine.