The cleanup lessons — WordPress and Drupal — get a compromised site safe again. This lesson answers the harder question: what actually happened? Cleanup without investigation is how sites get reinfected in a week — you mopped the floor without finding the leak. Investigation means preserving evidence, reconstructing the timeline, identifying the entry vector, and only then declaring the incident closed. It’s also what you owe anyone whose data may have been exposed, and what an insurer or lawyer will ask for if it comes to that.

Fair warning on scope: this is the deep end of Shop Class. If the site handles payment data or the stakes are legal, do this with a professional, not instead of one — the method below is also roughly what you’d be paying for, so you’ll be an informed client.

Preserve first — the evidence is more valuable than the uptime

The instinct is to start deleting suspicious files. Resist it: every deletion destroys timeline evidence. Before changing anything, capture the crime scene:

mkdir ~/incident-$(date +%Y%m%d) && cd ~/incident-$(date +%Y%m%d)
tar -czf docroot-snapshot.tar.gz /var/www/html
cp -r /var/log/apache2 ./logs-apache || cp -r /var/log/nginx ./logs-nginx
mysqldump -u USER -p DATABASE | gzip > db-snapshot.sql.gz

Pull the archive off the server (evidence stored on a compromised machine is evidence the attacker can edit). If the site must come down meanwhile, put up a static maintenance page — don’t restore from backup yet, because you don’t know which backups are clean and restoring overwrites the scene.

Build the file timeline — recently changed is recently touched

Attackers modify files; filesystems remember when:

find /var/www/html -type f -mtime -30 -printf "%T@ %TY-%Tm-%Td %TH:%TM %p\n" | sort -n

Read the last 30 days of modifications and mark what you can account for (deploys, uploads, cache files). What remains is the suspect list. Cluster timestamps tell stories: six files modified within the same minute, weeks after the last legit deploy, is the fingerprint of an automated infection. Note the earliest suspicious timestamp — that’s your candidate compromise time, and the pivot for everything that follows. (Caveat: attackers can forge mtimes; on ext4, debugfs crtime checks can catch that, but treat the timeline as strong evidence, not gospel.)

Hunt webshells by behavior, not just signatures

A webshell is a file that turns a web request into command execution. Signature grep catches the lazy ones:

grep -rEln "eval\s*\(\s*(base64_decode|gzinflate|str_rot13)|assert\s*\(\s*\\\$_(POST|GET|REQUEST)|system\s*\(\s*\\\$_" /var/www/html --include="*.php"

But the good ones hide, so hunt by anomaly: PHP files where no PHP belongs (uploads/, files/, theme image folders), files whose names imitate neighbors (wp-cache.php, class-walker.php, settings2.php), and integrity failures against known-good sources — wp core verify-checksums on WordPress; on Drupal, diff contrib against a fresh composer install into a scratch directory. Every confirmed webshell goes in the evidence log with its path and mtime: these are your Indicators of Compromise, and their timestamps extend the timeline.

Interrogate the access logs — the attack wrote its own confession

Web server logs are the closest thing to a security camera. First, pull every request to each webshell you found:

grep -h "settings2.php" logs-apache/access.log* | sort

This yields the attacker’s IPs and the first access time. Then pivot: what else did those IPs touch?

grep -hE "^(1\.2\.3\.4|5\.6\.7\.8) " logs-apache/access.log* | sort -k4

Now look before the first webshell hit for the entry itself. The patterns that answer “how did they get in”: a POST to a plugin/module path shortly before the first shell write (vulnerability exploitation — note the exact path and look up its CVE), hundreds of failed logins then a success (credential brute force), a single clean admin login from a strange IP with no failures (stolen or reused password — assume the credential is burned everywhere it was reused), or uploads through a legitimate form that accepted more than it should. Log rotation means the window is finite — which is an argument for shipping logs off-box, filed for after the incident.

Establish the blast radius honestly

With timeline and vector in hand, answer the exposure questions in writing: What could the attacker read (the database credentials in settings.php/wp-config.php mean assume the full database — users, emails, password hashes)? What did they change (content injections, new admin accounts, scheduled tasks)? Did anything leave (large or repeated outbound responses in the logs; on commerce sites, whether checkout templates were modified — card skimming is the money motive)? Check persistence beyond the docroot: crontabs, unfamiliar SSH keys in ~/.ssh/authorized_keys, database-stored triggers or malicious admin users. “We don’t know” is an acceptable answer only after you’ve looked.

Close the actual door, then rebuild forward

Now — and only now — cleanup, with the vector knowledge making it surgical instead of superstitious. The order: patch or remove the exploited component first (cleaning before patching is inviting the same script back), rotate every credential the attacker could have read (CMS admins, database, SSH, API keys — from a machine you trust), then rebuild the docroot from known-good sources — fresh core and packages, your version-controlled custom code, and uploads audited (no PHP in file trees, media only). Prefer rebuilding forward over restoring a backup unless the backup provably predates the earliest IOC timestamp; restoring a backdoored backup is the classic reinfection story.

Write the report while the logs are fresh

One page, plain language, evidence-backed: compromise date, entry vector, what the attacker did, what data was exposed to what degree of certainty, what was cleaned, what was rotated, what now prevents recurrence. This document is for the client, the insurer, any users you have a duty to notify — and for you in six months when a similar site knocks. Tape every show — incidents especially.

Set the tripwires that make next time shorter

The investigation’s parting gifts, installed while the pain is fresh: file-integrity monitoring (even a nightly find -mtime diff emailed somewhere), off-server log shipping so rotation stops eating evidence, login attempt alerting, and uptime monitoring that would notice the defacement variant. Verified working when you can answer: if the same attack ran tonight, what would page us, and how much timeline would we have tomorrow? If the honest answer is still “nothing and none” — that’s what the maintenance practice exists for, and after this lesson you know exactly what you’d be buying.