ops-ias-fraud-ip-ingest

Objective

In order to avoid serving ads to fraudulant traffic we use data of a third party ad verification provider called Integral Ad Science (IAS).

They are sending lists of IPs claimed as fraudulent by them on an hourly base (S3 upload). We will use that data to block the related traffic (or at least monitor the outcome in the first phase).

Architecture

Draw.io source

Resources Workflow

• IAS uploads a list of ~2.5Mio. IP's to a S3 bucket on hourly basis
• A StepFunction / Lambda ingests these IP's to the Somquery Redis Cache
  • in bunches of 100,000 in order to avoid latency in Redis / Timeouts of Lambda
  • concurrent ingest of a newer set before switching versions
  • a separate Lambda for removing entries of older versions
• Somquery is checking the remote IP of every call that is to return any VAST
  • the result in written into the debug output / Cerberus log data
  • the log data is monitored in Sherlog in order to get a figure of the ratio that would really get blocked

Ingest / Request Workflow

• After every IAS upload a timestamp is generated (<new-timestamp>)
• Ingest of one entry per IP with schema ip:<new-timestamp>:<the-bad-ip>
• After Ingest set ip:currentTimestamp to <new-timestamp>
• Another Lambda is constanty removing IP entries with timestamps oder than currentTimestamp
• Somquery has to perform 2 Redis calls:
  • const ts = await client.get('ip:currentTimestamp);
  • const isFraud = await client.get(`ip:${ts}:${<ip-to-be-checked>}`);

Q & A

Why ingest into Redis with bunches of only 100,000 IPs?

Redis is a highly available mem cache used in all our Somquery EC2 instances. Operating with bigger chunks could potentially increase any risk of interfere with Redis' life performance or of a timeout of the Lambda function.

Why a separate Lambda for deleting IP entries of older versions?

Basicly for the same reasons (performance / timeout) plus an additional security layer that not any older data caused by a Ingest Lambda crash would bloat up Redis' size without noticing.

AWS Resources

• Bucket: arn:aws:s3:::adtech-ias-data-exchange
• S3 Event Rule: arn:aws:events:eu-central-1:377710842884:rule/ias-data-exchange-event-rule
• StepFunction: arn:aws:states:eu-central-1:377710842884:stateMachine:IasIngestFraudIpsStateMachine
• Ingest Lambda: arn:aws:lambda:eu-central-1:377710842884:function:ias-ingest-fraud-ips
• Cleanup Trigger: arn:aws:events:eu-central-1:377710842884:rule/ias-fraud-data-cleanup-hourly
• Cleanup Lambda: arn:aws:lambda:eu-central-1:377710842884:function:ias-cleanup-fraud-ips