Unable to record Shoutcast

[Star6key]

Last year on the T4 I was able to record songs on Shoutcast using the plugin (I've still got over 300 songs on the drive). Just tried again and get some sort of 'Forbidden' error (not sure of the actual message as it only stays on screen for a split second). Tried a number of different stations with the same result. This happens on both the U/T4.

Any ideas?

fw beyonwiz 17.5.20190207 (17.5-819-gdd3f1fa)

[prl]

Any ideas?

Could the error message be: "error -56 [HTTP:403 - Access Forbidden (try changing the UserAgent)]"?

[Star6key]

That's the one.

[prl]

Good!

That error message comes from the streamripper application that does the actual recording for SHOUTcast.

When I run streamripperfrom the commandline with the arguments used to make an actual recording, I see:

Code: Select all

root@beyonwizt4:/media/hdd/logs# /usr/bin/streamripper http://54.38.186.141:80 -d /hdd/streamripper -r 9191 -l 120
Connecting...

error -56 [HTTP:403 - Access Forbidden (try changing the UserAgent)]
bye..
shutting down
root@beyonwizt4:/media/hdd/logs#

[-d: destination directory, -r: use relay port, -l: length of recording in seconds]

If I supply a fake UserAgent using the -u option, it works:

Code: Select all

root@beyonwizt4:/media/hdd/logs# /usr/bin/streamripper http://54.38.186.141:80 -d /hdd/streamripper -r 9191 -l 120 -u fakeagent
Connecting...
stream: UpDance.de // Fresh MIX!
server name: SHOUTcast/
declared bitrate: 32
meta interval: 16384
relay port: 9191
[getting track name... ]
Skipping...   ] Hubu.FM - Live [   17kb]
...
shutting down
bye..
root@beyonwizt4:/media/hdd/logs#

The server being connected to doesn't seem to like the string "ripper" in the user agent name (the default agent name is "Streamripper/1.x"):

Code: Select all

root@beyonwizt4:/media/hdd/logs# /usr/bin/streamripper http://54.38.186.141:80 -d /hdd/streamripper -r 9191 -l 120 -u RipperRita/1.1
Connecting...

error -56 [HTTP:403 - Access Forbidden (try changing the UserAgent)]
bye..
shutting down
root@beyonwizt4:/media/hdd/logs# /usr/bin/streamripper http://54.38.186.141:80 -d /hdd/streamripper -r 9191 -l 120 -u JackTheRipper/1.1
Connecting...

error -56 [HTTP:403 - Access Forbidden (try changing the UserAgent)]
bye..
shutting down
root@beyonwizt4:/media/hdd/logs#

This looks like a (fairly lazy) attempt on the part of the server to discourage recording.

It's probably easy to work around by using the -u option in the Beyonwiz UI code, but there's a question about whether the servers' apparent intentions should be respected.

I only tested using one stream's server. Workarounds that apply to that stream may or may not work on other streams.

[peteru]

It's probably easy to work around by using the -u option in the Beyonwiz UI code

Done.

[Star6key]

It's probably easy to work around by using the -u option in the Beyonwiz UI code

Done.

Excellent.

[beyonbloke]

Could someone explain how I transfer this ammended streamripper script/code to the beyonwiz. I'm still an "apprentice".
I've got the dongle in and filezilla set up ok. Do I copy the ammended script into notepad and transfer to a particular directory on the T4 via filezilla, or am I going down the wrong rabbit hole here?

Thanks
Beyonbloke

[beyonbloke]

Wrong rabbit hole I've discovered. Over and out.

beyonbloke

[prl]

It is possible to hack your T4 to fix the bug, but it involves extracting the source code for one of the modules in the SHOUTcast plugin, and copying that to the right place on the T4. If you really want to try to do it, I can give instructions, but the simplest thing to do is to wait for the next firmware upgrade, which will have the fix.

[beyonbloke]

If you really want to try to do it, I can give instructions, but the simplest thing to do is to wait for the next firmware upgrade, which will have the fix.

Actually, I wouldn't mind having a go if you could give me a heads up how to go about it.

beyonbloke

[adoxa]

Extract plugin.pyo from the attachment, copy it to /usr/lib/enigma2/python/Plugins/Extensions/IniSHOUTcast (should be no problem just overwriting it, but perhaps rename first) and restart.

[Star6key]

Just followed Adoxa's instructions (trying to record 'smoothlounge.com global'), and got the same error message -

error -56 [HTTP:403 - Access Forbidden (try changing the UserAgent)]

I then tried another station, but this time it started recording. The same for a few others. Seems there are restrictions for certain stations.

[beyonbloke]

[quoteExtract plugin.pyo from the attachment, copy it to /usr/lib/enigma2/python/Plugins/Extensions/IniSHOUTcast (should be no problem just overwriting it, but perhaps rename first) and restart.][/quote]

Will give it a go and report back. Thanks.
beyonbloke

[Grumpy_Geoff]

Just followed Adoxa's instructions (trying to record 'smoothlounge.com global'), and got the same error message -

error -56 [HTTP:403 - Access Forbidden (try changing the UserAgent)]

I then tried another station, but this time it started recording. The same for a few others. Seems there are restrictions for certain stations.

Before I applied the updated plugin, I tried the example station from prl - UpDance.de (Fresh MIX!). It errored as expected.
I then updated the plugin, restarted the UI, and tried again - this time with success.

I then tried your example station, SmoothLounge.Com Global Radio, and got the error as you did.
I then tried the same station via the command line, using the 'fakeagent' useragent as per prl's previous examples and had success -

Code: Select all

root@beyonwizu4:/hdd/streamripper# /usr/bin/streamripper http://149.56.155.210:80 -d /hdd/streamripper -l 240 -u fakeagent
Connecting...
stream: SmoothLounge.com Global
server name: SHOUTcast/
declared bitrate: 128
meta interval: 16384

[skipping...   ] Sabo & Zeb - Cool NY [  2.58M]
[ripping...    ] Curtis Macdonald - Ancient Souls [  927kb]
Time to stop is here, bailing
shutting down
bye..

I believe the update to the plugin may be using 'IniSHOUTcast/1.0' as the useragent name so I tried that, no go -

Code: Select all

root@beyonwizu4:/hdd/streamripper# /usr/bin/streamripper http://149.56.155.210:80 -d /hdd/streamripper -l 240 -u 'IniSHOUTcast/1.0'
Connecting...

error -56 [HTTP:403 - Access Forbidden (try changing the UserAgent)]
bye..
shutting down
root@beyonwizu4:/hdd/streamripper#

I changed the useragent name and had some success, so it appears some stations don't like 'shoutcast' in the agent name -

Code: Select all

root@beyonwizu4:/hdd/streamripper# /usr/bin/streamripper http://149.56.155.210:80 -d /hdd/streamripper -l 240 -u 'IniSHOUT/1.0'
Connecting...
stream: SmoothLounge.com Global
server name: SHOUTcast/
declared bitrate: 128
meta interval: 16384

[skipping...   ] Nuvo - Do It Again [  3.02M]
[ripping...    ] Rick Callier - My Guitar [  472kb]
Time to stop is here, bailing
shutting down
bye..
root@beyonwizu4:/hdd/streamripper#
root@beyonwizu4:/hdd/streamripper# /usr/bin/streamripper http://149.56.155.210:80 -d /hdd/streamripper -l 240 -u 'IniSHOUTcaster/1.0'
Connecting...

error -56 [HTTP:403 - Access Forbidden (try changing the UserAgent)]
bye..
shutting down
root@beyonwizu4:/hdd/streamripper# /usr/bin/streamripper http://149.56.155.210:80 -d /hdd/streamripper -l 240 -u 'IniSHOUT-cast/1.0'
Connecting...
stream: SmoothLounge.com Global
server name: SHOUTcast/
declared bitrate: 128
meta interval: 16384

[skipping...   ] Sergio in Acapulco - Loungin [  2.75M]
[ripping...    ] Simone - Don't Know What [  733kb]
Time to stop is here, bailing
shutting down
bye..
root@beyonwizu4:/hdd/streamripper#

[prl]

I believe the update to the plugin may be using 'IniSHOUTcast/1.0' as the useragent name so I tried that, no go -

That's the correct User-Agent string for the updated plugin.

I changed the useragent name and had some success, so it appears some stations don't like 'shoutcast' in the agent name -

It doesn't entirely surprise me that the change doesn't work for all stations. I never tested that specific user agent name, and I only tried the workaround on one stream server:

I only tested using one stream's server. Workarounds that apply to that stream may or may not work on other streams.

An alternative would be to use the same User-Agent string as is used by the gstreamer stream player, Enigma2 HbbTV/1.1.1 (+PVR+RTSP+DL;Beyonwiz;;;)*. That would mean that if the SHOUTcast plugin can play it, it should also be able to record it.

Code: Select all

root@beyonwizv2:/media/hdd/logs# /usr/bin/streamripper http://149.56.155.210:80 -d /hdd/streamripper -l 240 -u 'Enigma2 HbbTV/1.1.1 (+PVR+RTSP+DL;Beyonwiz;;;)'
Connecting...
stream: SmoothLounge.com Global
server name: SHOUTcast/
declared bitrate: 128
meta interval: 16384

skipping...   ] Sine - Always [   49kb]

If that UA string is to be used, it should probably be constructed in the same way that gstreamer constructs it, rather than just copy/pasting the bits.

But as I noted in that same post:

This looks like a (fairly lazy) attempt on the part of the server to discourage recording.

It's probably easy to work around by using the -u option in the Beyonwiz UI code, but there's a question about whether the servers' apparent intentions should be respected.

* I had to Wireshark it. That also led me to finding out how port mirroring works on my network switch

[peteru]

* I had to Wireshark it. That also led me to finding out how port mirroring works on my network switch

You can capture the packets on the Beyonwiz itself. If you prefer the Wireshark GUI over the command line interface, you can always capture to file and load that packet capture into Wireshark. I don't have any smart switches, so I've had to do that before.

[beyonbloke]

SUCCESS!!!!
Well, it can record most streams amyway which is fine for me.

Thanks to "adoxa" for the guide.

beyonbloke

[prl]

* I had to Wireshark it. That also led me to finding out how port mirroring works on my network switch

You can capture the packets on the Beyonwiz itself. If you prefer the Wireshark GUI over the command line interface, you can always capture to file and load that packet capture into Wireshark. I don't have any smart switches, so I've had to do that before.

Thanks. I wondered whether there was a wireshark plugin, but I didn't check. I had Wireshark on the Mac anyway, and I learnt something new that way

[peteru]

It's not a Wireshark plugin, just good old pcap with tcpdump.

[adoxa]

Updated the previous attachment, using Enigma2 HbbTV/1.1.1 (+PVR+RTSP+DL;Beyonwiz;;;) as the user agent. It's hardcoded in lib/base/httpstream.cpp, lib/service/servicemp3.cpp and lib/python/Tools/Downloader.py (prepending machine brand & name and using OpenVix instead of Beyonwiz), so I hardcoded it here, too.

[prl]

It's not a Wireshark plugin, just good old pcap with tcpdump.

I'd have had to read more documentation, then.

[Star6key]

Updated the previous attachment, using Enigma2 HbbTV/1.1.1 (+PVR+RTSP+DL;Beyonwiz;;;) as the user agent. It's hardcoded in lib/base/httpstream.cpp, lib/service/servicemp3.cpp and lib/python/Tools/Downloader.py (prepending machine brand & name and using OpenVix instead of Beyonwiz), so I hardcoded it here, too.

That did the trick. Smoothlounge now records.
Thanks Adoxa & Prl.

[prl]