Performance Problems

[Carl Hibbard]

We seem to be under massive usage maybe AI crawlers hard to be sure but it is proving difficult to keep the smallish server running.

[Carl Hibbard]

We seem to be under massive usage maybe AI crawlers hard to be sure but it is proving difficult to keep the smallish server running.

For now I am blocking everything BUT the United Kingdom which is really excessive.

[Carl Hibbard]

A unique visitors of 81.78k per hour which is twice normal is simply too much.

[Roger de Coverly]

A unique visitors of 81.78k per hour which is twice normal is simply too much.

Another pHp site I visit had similar problems. It was claimed this was AI bots looking for content.

[Carl Hibbard]

A unique visitors of 81.78k per hour which is twice normal is simply too much.

Another pHp site I visit had similar problems. It was claimed this was AI bots looking for content.

The problem is not all AI bots follow the rules or even identify themselves properly with Singapore and Brazil being our top countries looking currently.

[Carl Hibbard]

We seem to be under massive usage maybe AI crawlers hard to be sure but it is proving difficult to keep the smallish server running.

For now I am blocking everything BUT the United Kingdom which is really excessive.

I will shortly disable this restriction to see what happens but this is probably more fingers crossed than anything technical.

[Carl Hibbard]

The current mitigations seem to be holding.

[John Clarke]

You acted wisely Carl, even though it was a bit disturbing to find this morning (NZ time) that I couldn't get in. I was wondering if it might be a hack or a DoS of some kind. Nothing else made much sense.

[Carl Hibbard]

You acted wisely Carl, even though it was a bit disturbing to find this morning (NZ time) that I couldn't get in. I was wondering if it might be a hack or a DoS of some kind. Nothing else made much sense.

I suspect this AI Crawlers issue to be an ongoing problem.

[Christopher Kreuzer]

https://www.cloudflare.com/en-gb/press- ... -at-large/

Cloudflare Just Changed How AI Crawlers Scrape the Internet-at-Large; Permission-Based Approach Makes Way for A New Business Model

Hopefully that step will also help (though I do not know enough about this site to know how the backend works, I have heard similar stories in other forums).

[Carl Hibbard]

I doubt our problem which is a US company with data coming out of Singapore for some reason will be covered by any rules.

[Carl Hibbard]

Happening again not sure what I am missing so had to go block all but GB excessive.

[Carl Hibbard]

The block has been changed again but the investigation into the issues is ongoing I am afraid.

[John Clarke]

Ah well, if this is set to be an ongoing issue, then I'd better bid you all a temporary farewell. I've greatly enjoyed hanging out here over the years, and have (I hope) been able to make some worthwhile contributions. But I guess all things must pass.

Should anyone want to reach me, then email [email protected] (a PM is pointless, of course).

Over and out, at least for now. John C

[Carl Hibbard]

We are for now back online John hopefully I will get to the root of the problem.