English Chess Forum

We seem to be under massive usage maybe AI crawlers hard to be sure but it is proving difficult to keep the smallish server running.

Carl Hibbard wrote: ↑

Mon Jul 14, 2025 7:34 pm

We seem to be under massive usage maybe AI crawlers hard to be sure but it is proving difficult to keep the smallish server running.

For now I am blocking everything BUT the United Kingdom which is really excessive.

A unique visitors of 81.78k per hour which is twice normal is simply too much.

Carl Hibbard wrote: ↑

Mon Jul 14, 2025 7:49 pm

A unique visitors of 81.78k per hour which is twice normal is simply too much.

Another pHp site I visit had similar problems. It was claimed this was AI bots looking for content.

Roger de Coverly wrote: ↑

Mon Jul 14, 2025 9:43 pm

Carl Hibbard wrote: ↑

Mon Jul 14, 2025 7:49 pm

A unique visitors of 81.78k per hour which is twice normal is simply too much.

Another pHp site I visit had similar problems. It was claimed this was AI bots looking for content.

The problem is not all AI bots follow the rules or even identify themselves properly with Singapore and Brazil being our top countries looking currently.

Carl Hibbard wrote: ↑

Mon Jul 14, 2025 7:47 pm

Carl Hibbard wrote: ↑

Mon Jul 14, 2025 7:34 pm

We seem to be under massive usage maybe AI crawlers hard to be sure but it is proving difficult to keep the smallish server running.

For now I am blocking everything BUT the United Kingdom which is really excessive.

I will shortly disable this restriction to see what happens but this is probably more fingers crossed than anything technical.

The current mitigations seem to be holding.

You acted wisely Carl, even though it was a bit disturbing to find this morning (NZ time) that I couldn't get in. I was wondering if it might be a hack or a DoS of some kind. Nothing else made much sense.

John Clarke wrote: ↑

Tue Jul 15, 2025 8:30 am

You acted wisely Carl, even though it was a bit disturbing to find this morning (NZ time) that I couldn't get in. I was wondering if it might be a hack or a DoS of some kind. Nothing else made much sense.

I suspect this AI Crawlers issue to be an ongoing problem.

https://www.cloudflare.com/en-gb/press- ... -at-large/

Cloudflare Just Changed How AI Crawlers Scrape the Internet-at-Large; Permission-Based Approach Makes Way for A New Business Model

Hopefully that step will also help (though I do not know enough about this site to know how the backend works, I have heard similar stories in other forums).

I doubt our problem which is a US company with data coming out of Singapore for some reason will be covered by any rules.

Happening again not sure what I am missing so had to go block all but GB excessive.

The block has been changed again but the investigation into the issues is ongoing I am afraid.

Ah well, if this is set to be an ongoing issue, then I'd better bid you all a temporary farewell. I've greatly enjoyed hanging out here over the years, and have (I hope) been able to make some worthwhile contributions. But I guess all things must pass.

Should anyone want to reach me, then email [email protected] (a PM is pointless, of course).

Over and out, at least for now. John C

We are for now back online John hopefully I will get to the root of the problem.