Dealing with Unknown User Agent clients

[cobaltcosmic]

Hi all, I am a brand new RadioDJ user and (thanks Marius!) have a question about a general streaming issue that maybe someone else has dealt with and can help me understand what I am seeing.  My station is listed on xiph.org and internet-radio.com only.

First, my setup:

Dell R320 (overkill) dedicated server running Windows 10, behind a firewall
Running:
-  IceCast2
-  Sound Empire Caster 1.2.7
-  RadioDJ 2.0.0.6 Free


The issue:

To date, I keep seeing regular connections from IP addresses that report their client as "Unknown".  These IPs all are coming from Google Cloud IP blocks.  If I search Google for the IP, I do find some occasional sightings of the IPs on other radio stations connected client list.  But I can't see where this would be an actual user tuning in, so what is it?  What is Google doing with my stream?


-Thanks!

Andrew

[stevewa]

assuming you are tlaking about the icecast access logs? how long is the duration of their connection? are they only short duration pings? are they for specific urls? they might be DOS crack attempts. have you tried using deny-ip list to block those worrisome ip addresses? <deny-ip>/path_to_ip_denylist</deny-ip>

[cobaltcosmic]

Hi Stevewa,

Yes, the access logs.  I have only one mountpoint, and yes, these specific clients are requesting access to that mountpoint.  I do see instances of connection attempts to the root mountpoint, but for obvious reasons those clients don't show a long connection time in the Client Access List web page.  What baffles me is these clients have the longest connection times (2-8 hour timeframes).  Why are all these clients from Google Cloud the ones with the longest connection times?

Yes, I have setup the deny list and have added some of these clients, but a new client from within those IP ranges appears within hours of blocking the previous one.  What I want to avoid is a stream-ripper or worse, a re-streamer.

-Andrew

[stevewa]

you can use  max-listener-duration  to set a max amount that a listener can stay connected.

you can use <authentication type="url"> and <option name="listener_add" value="http://auth.example.org/listener_joined.php"
to have a php script check the ip address of the listener connection request, to see if it's subnet is within the same subnet as those google farms, and prevent access.

[pirata]

Hi there!

If you are using Icecast on Linux, just have a look under:
/usr/share/icecast2/web

or

/etc/icecast2/web/

if there's a file named robots.txt edit it and paste into the file this text:

User-Agent: *
Allow: /

# go away Googlebot
User-agent: Googlebot
Disallow: /

if robots.txt isn't there, just create it and put those lines into it.

That will help you keeping Google bots away from you

Flavio

[cobaltcosmic]

Stevewa,

I see that I would need to setup the stream beyond the default config.  However, I read elsewhere that some players will re-connect...  Although, I'm not really worried about a real listener.  I'll give this a try and see what happens.  Otherwise, I could go the script route you mentioned. 

Pirata,

I am running IceCast on windows, but there was a WEB folder.  I added a robots.txt file as you suggested.  So far though I am still seeing connections from those IP ranges.

Thanks!
-Andrew

[stevewa]

while doing consulting work for a client, I noticed via the Centovacast interface, persistent connections from russia. the ip address is 185.62.102.73, and the user-agent details says this is a company name radio.tools
it appears they have bots which are "listening" to thousands of streams all over the world, trying to collect metadata on songs played, so they can create reports and sell them to subscribers. i will be adding their ip address to the block list.