← Wall of Shame · Well-behaved crawlers
Parallel Web Systems
verified
User agent matched Parallel Web Systems's crawler and the source addresses passed Parallel Web Systems's published verification. These requests fetched paths disallowed by our robots.txt.
Networks seen: AS396982 GOOGLE-CLOUD-PLATFORM.
Totals by window
| Window | Hours held | Pages | Max depth | Bytes | Sessions | Read the rules | First seen | Last seen |
|---|---|---|---|---|---|---|---|---|
| 24h | 0.00 | 0 | 0 | 0 B | 0 | 0 | - | - |
| 7d | 0.00 | 0 | 0 | 0 B | 0 | 0 | - | - |
| 30d | 0.00 | 0 | 0 | 0 B | 0 | 0 | - | - |
| all | 152.66 | 10822 | 7 | 23.9 MB | 329 | 0 | 2026-09-29 08:22Z | 2026-10-02 21:43Z |
"Read the rules" counts sessions that fetched robots.txt before their first disallowed page.
Observed traits
What these clients sent and how they walked the maze, from the raw logs we keep (the last 90 days). Each line is an observation, not a verdict on who or what a client is (how they are measured). A client is one address with one user agent; 10 clients here.
| Observed | Clients |
|---|---|
| Sent no Accept-Language header. Mainstream browsers always send one. | 5 |
| Came from a network that X4BNet's public list classifies as datacenter or hosting. | 10 |
Fastest pace: 12 disallowed pages within one minute from one client.
TLS fingerprints (JA4), most used first: t13d1711h2_5b57614c22b0_5894756fee65 (6).
Daily
Pages per day, last 90 days (peak 6412).
| Day (UTC) | Pages | Hours held | Bytes | Sessions |
|---|---|---|---|---|
| 2026-10-02 | 28 | 0.38 | 62.3 kB | 6 |
| 2026-09-30 | 6412 | 90.58 | 14.1 MB | 220 |
| 2026-09-29 | 4382 | 61.70 | 9.8 MB | 103 |
Top user agents
| User agent | Pages |
|---|---|
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ShapBot/0.1.0 | 10822 |
Networks
Verified crawler addresses are shown individually.
23.251.146.115/32 34.9.172.22/32 34.31.203.120/32 34.44.142.114/32 34.44.196.215/32 34.45.55.180/32 34.46.138.166/32 34.68.39.29/32 34.70.129.79/32 34.122.173.216/32
Sample paths
Recent disallowed URLs fetched, most recent first.
/lawn/a4kzbmc3rqfyjwmvlc5huhzf/lawn/a53q5equ7mpd6vvvxg5huhzf/lawn/a5zq3yxdv3chvz2ndk5huhzf/lawn/a53q5equ7mpd6vvvxg5huhzf/lawn/1957/notes/a47fvxjpwlnkfnyij65huhzf
Generated 2026-11-02 10:03 UTC.