Provider visiting - keeping the FTA EPG current

[Grumpy_Geoff]

Version 1.2:
The AutoVisit script has been re-worked and it now uses streaming for both live and standby modes, instead of the previous methods of zapping (when in standby) or recording (live/running).
Because zapping is no longer used, it's also not restricted to using the current TV bouquet (which may not have been your usual bouquet and thus not have had all of your providers, missing out scanning some of them). All providers are now scanned as they're taken from the Terrestrial TV LCN bouquet.
Streaming is a lot gentler on the recording drive too.
There's no longer a need to adjust the number of providers in your broadcast area, as the script works it out.

So, all in all, the new version is a lot simpler and less intrusive

The simplest way to prove AutoVisit has updated the EPG is, prior to your first run of AutoVisit, enable IceTV and then disable it as this will clear the EPG of data. Then bring up the EPG and if you're real quick it'll be empty then after a few seconds you'll see it has populated for the current provider/broadcaster, or if you're a bit slower to open the EPG then you'll see that provider's guide data only. If you've got any recordings running then the guide data will be populated for those providers too.
Then let AutoVisit run and check the EPG and you'll find now its full of data

Above copied from the edit in the opening post.
The new version is attached as version 1.2 in the original post.

[ETA_2016]

I updated to version 1.2 and now AutoVisit won't run, due to not detecting the AutoVisit.allow file.

I have triple checked that the flags directory is correct, and updated enigma2_pre_start.sh with the correct directory for AutoVisit.

I have no idea why it's not working, and it's not generating any errors. A radio service, in the Favourites (Radio) bouquet, is active.

[Grumpy_Geoff]

I updated to version 1.2 and now AutoVisit won't run, due to not detecting the AutoVisit.allow file.

I have triple checked that the flags directory is correct, and updated enigma2_pre_start.sh with the correct directory for AutoVisit.

I have no idea why it's not working, and it's not generating any errors. A radio service, in the Favourites (Radio) bouquet, is active.

Ooh, that's not nice.

So there's nothing reported in the AutoVisit_errs.log file?

Can you comment out the "exec output redirection" on line 17 by putting a # at the beginning of the line.
Then run the script from the command line and see if anything shows. Run it either way -
From the same directory as the script:

Code: Select all

./AutoVisit.sh

or from another directory:

Code: Select all

/media/hdd/dir-to-run-stuff/AutoVisit.sh

If it actually runs, use CTRL-C to kill it (but it sounds like it won't).
It sounds like there's a script error and the script is falling through the loop and it hits the "no allow file found" line.

[ETA_2016]

I updated to version 1.2 and now AutoVisit won't run, due to not detecting the AutoVisit.allow file.

I have triple checked that the flags directory is correct, and updated enigma2_pre_start.sh with the correct directory for AutoVisit.

I have no idea why it's not working, and it's not generating any errors. A radio service, in the Favourites (Radio) bouquet, is active.

...
So there's nothing reported in the AutoVisit_errs.log file?

No.

Can you comment out the "exec output redirection" on line 17 by putting a # at the beginning of the line.
Then run the script from the command line and see if anything shows. Run it either way -
From the same directory as the script: ...

Using telnet, I ran the following command:

Code: Select all

/media/hdd/plugins+skins/AutoVisit.sh

and, the output was the same as the log files:

AutoVisit.allow file not detected
Exiting at: 01/06/20 18:02:20

Was this what you wanted?

[Grumpy_Geoff]

I ran the following command using telnet:

Code: Select all

/media/hdd/plugins+skins/AutoVisit.sh

and the output was the same as the log files:

AutoVisit.allow file not detected
Exiting at: 01/06/20 18:02:20

Was this what you wanted?

So just those two lines?
No header lines, free tuner count, like this ?

Code: Select all

-----------------
dd/mm/yy hh24:mi:ss
Free tuners: n
AutoVisit.allow file not detected
Exiting at: dd/mm/yy hh24:mi:ss

Try this -
sh -x /media/hdd/plugins+skins/AutoVisit.sh

Are you sure about the flag files location specification?

[ETA_2016]

Sorry, for wasting your time; I just found that the flag files directory included /AutoVisit.sh!

It's now fixed, and ran successfully, but produced the following errors:

Redirecting output to 'wget-log.9'.

Redirecting output to 'wget-log.10'.

Redirecting output to 'wget-log.11'.

Redirecting output to 'wget-log.12'.

Redirecting output to 'wget-log.13'.

[Grumpy_Geoff]

It's now fixed, and ran successfully, but produced the following errors:

Redirecting output to 'wget-log.9'.

Redirecting output to 'wget-log.10'.

Redirecting output to 'wget-log.11'.

Redirecting output to 'wget-log.12'.

Redirecting output to 'wget-log.13'.

From the command line right, and with the output redirection commented out?
Those 'Redirecting output to' lines are from the stream command.
You shouldn't see them when it's run the "usual" way (at least I don't).

[ETA_2016]

It's now fixed, and ran successfully, but produced the following errors:

Redirecting output to 'wget-log.9'.

Redirecting output to 'wget-log.10'.

Redirecting output to 'wget-log.11'.

Redirecting output to 'wget-log.12'.

Redirecting output to 'wget-log.13'.

From the command line right, and with the output redirection commented out?
Those 'Redirecting output to' lines are from the stream command.
You shouldn't see them when it's run the "usual" way (at least I don't).

No, these errors were in the errors log file. I'm running AutoVisit normally (in the background, with the output to a log file).

I just wanted to check that the errors were not unexpected.

[Grumpy_Geoff]

No, these errors were in the errors log file. I'm running AutoVisit normally (in the background, with the output to a log file).

Hmm, my 'AutoVisit_errs.log' file is empty.

[ETA_2016]

No, these errors were in the errors log file. I'm running AutoVisit normally (in the background, with the output to a log file).

Hmm, my 'AutoVisit_errs.log' file is empty.

The errors log file was empty, then updated, one second after the AutoVisit run start time, in the AutoVisit log file.

Don't stress. I'll let it run a few times, to see if those errors keep reoccurring. I'm now have AutoVisit running hourly.

[ETA_2016]

No, these errors were in the errors log file. I'm running AutoVisit normally (in the background, with the output to a log file).

Hmm, my 'AutoVisit_errs.log' file is empty.

...

Don't stress. I'll let it run a few times, to see if those errors keep reoccurring...

No new errors have been produced. So, AutoVisit is working fine and not producing any errors.

Thanks, for all your work on this project.

[Grumpy_Geoff]

No new errors have been produced. So, AutoVisit is working fine and not producing any errors.

Good.

Thanks, for all your work on this project.

Sure. Do you stream live TV/radio much from your T4?
I now know how to detect that live streaming, and it should be taken into account when determining the number of in-use tuners.
I've made that change and I'll attach it if you do.

[ETA_2016]

No new errors have been produced. So, AutoVisit is working fine and not producing any errors.

Good.

Thanks, for all your work on this project.

Sure. Do you stream live TV/radio much from your T4?

No, I've never done that; nor do I need to.

I now know how to detect that live streaming, and it should be taken into account when determining the number of in-use tuners.
I've made that change and I'll attach it if you do.

I guess it's probably a good idea to update AutoVisit, in case anyone else who downloads it, does stream.

[Grumpy_Geoff]

I guess it's probably a good idea to update AutoVisit, in case anyone else who downloads it, does stream.

Sure, but I think you may be the only customer

[Henk]

I guess it's probably a good idea to update AutoVisit, in case anyone else who downloads it, does stream.

Sure, but I think you may be the only customer

Hello GG,

definitely a second customer here in Kalgoorlie!

Works perfectly with streaming services included in the Favorites.

[Grumpy_Geoff]

Hello GG,

definitely a second customer here in Kalgoorlie!

Works perfectly with streaming services included in the Favorites.

Hi Henk,

Okay, I'll attach the new version that takes into account current tuner use through live streaming when determining the number of in-use tuners.
Hopefully I can get to it on Wednesday, as I'm off to the WAFL tomorrow to hopefully watch West Perth beat South Fremantle, and if they do I'll have a hangover on Monday, and Tuesday we've a school holiday family outing.

[Grumpy_Geoff]

definitely a second customer here in Kalgoorlie!

Works perfectly with streaming services included in the Favorites.

Hi Henk,
Just for you

Updated first post with -
Version 1.3:
AutoVisit now detects any tuner use through current live TV/radio streaming, and takes that into account when deciding if there's enough spare tuner capacity for an AutoVisit run.

The new version is attached in that opening post.

[ETA_2016]

definitely a second customer here in Kalgoorlie!

Works perfectly with streaming services included in the Favorites.

Hi Henk,
Just for you

Updated first post with -
Version 1.3:
AutoVisit now detects any tuner use through current live TV/radio streaming, and takes that into account when deciding if there's enough spare tuner capacity for an AutoVisit run.

The new version is attached in that opening post.

What! What about me?

Just downloaded it. Thanks again, for all your work on this project.

[Grumpy_Geoff]

What! What about me?

What About Me
Moving Pictures
Well there's a little boy waiting at the counter of the corner shop
He's been waiting down there, waiting half the day,
They never ever see him from the top
He gets pushed around, knocked to the ground,
He gets to his feet and he says

[Chorus:]
What about me? It isn't fair
I've had enough, now I want my share
Can't you see, I want to live
But you just take more than you give


Well, as you know, you're the other 50% of my customers, and since you don't stream live I didn't specifically mention you as there wasn't any other functionality change...

Thanks, for all your work on this project.

Sure. Do you stream live TV/radio much from your T4?
I now know how to detect that live streaming, and it should be taken into account when determining the number of in-use tuners.
I've made that change and I'll attach it if you do.

No, I've never done that; nor do I need to.

...Sorry.

[Henk]

Hi Henk,
Just for you

Geoff,

Many thanks from the other 50%!

[adoxa]

I think this is causing my V2 to freeze (haven't had any issues with the T2):

Code: Select all

16/10/20 20:07:55
Free tuners: 4
Number of active service names: 0
Pattern match for existing tuning: _(dummy)_
Service reference: 1:0:19:2C5:243:1010
  - Provider/broadcaster's TSID:ONID is 243:1010
  - Streaming
Service reference: 1:0:16:787:327D:3271
...
Finished looping through services to process
Finished cycle: 16/10/20 20:10:26

-----------------
16/10/20 20:30:26
Free tuners: 3
Current recording service name/program: SBS VICELAND HD: How The Mp3 Changed Music
Number of active service names: 1
Active services: SBS VICELAND HD
Pattern match for existing tuning: 3B0:3202
Service reference: 1:0:19:2C5:243:1010
  - Provider/broadcaster's TSID:ONID is 243:1010
  - Streaming

That's where it stops (it was about 20:40, the clock froze at 20:30). Had to power cycle, which wiped the log (new one same time as the old one). I didn't notice anything in previous logs, though. I have preferred tuner set to C and recording tuner set to D (the dual USB tuner). It's possible this could just be my box (it's the sample), as fake-hwclock didn't work either, but it does work on the test V2. I'll experiment some more in a couple of days.

[Grumpy_Geoff]

So it appears, on the surface, that the streaming may have locked up.
It's a pity the log got clobbered.
I see the previous cycle had nothing but the streaming occupying the USB tuner, but with the "frozen" cycle the USB tuner was already recording when the stream was initiated.
Or it may be something else entirely.

[adoxa]

It's only happened during recording, so there may be something going on with the USB tuner and streaming. Plans for the weekend, so might not be till Monday that I can experiment.

[adoxa]

Managed to have a look this morning. It did seem to be wget/OWIF that was the problem - started, but wasn't downloading anything, then everything froze (including the shell). Wasn't able to reproduce it, though, so still not sure of the exact circumstances. I've changed it to only stream if there are no in-use tuners.

[Grumpy_Geoff]

Managed to have a look this morning. It did seem to be wget/OWIF that was the problem - started, but wasn't downloading anything, then everything froze (including the shell).

It wouldn't surprise me if OWIF has a memory leak.

I don't have a V2, but today I have had AutoVisit cycling through for hours on end on my U4 whilst watching and recording the Bathurst 1000.
The U4 was also streaming Bathurst 1000 (on "10") for hours on end to a tablet using wizOS. AutoVisit was cycling through at the same time.

That's really FUBARed if you couldn't even get a command prompt!

[Half Round]

Hi, I installed AutoVisit v1.3 on a V2 and T2. a few days ago
On the V2 yesterday something strange happened on the recordings.
firmware 19.3 Build 20201011 (beta)

The recording list had multiple entries for a single recording. Picture of screen, movie file list and autovisit log attached.

movies-file-list.txt

(21.91 KiB) Downloaded 90 times

AutoVisit.201018_120923.log

(36.87 KiB) Downloaded 97 times

The FBI program had one following the other, the first had 4 entries and the second was OK - a single entry.

Any ideas?

[Grumpy_Geoff]

Any ideas?

My guess is the system time on your Wiz has gone haywire. It went back roughly an hour just after 2pm -

Code: Select all

18/10/20 13:39:25
Finished cycle: 18/10/20 13:41:55

-----------------

18/10/20 14:01:55
Finished cycle: 18/10/20 13:04:18

Then it appears to have reset itself, then starts jumping forwards and backwards.
By the time it hits the wall-clock time for the timers to begin firing, it's jumping around and they're starting, stopping, and re-starting and stopping. Rinse, repeat.

[prl]

Is time set from the network (NTP) or from the broadcast signal?

[Half Round]

My guess is the system time on your Wiz has gone haywire. It went back roughly an hour just after 2pm -

Is time set from the network (NTP) or from the broadcast signal?

Hi Gents,

Thanks Geoff, I missed that trick. After reading your reply I went and checked the time setup. It was using station time so I changed it to NTP(internet) on both machines.

Thanks Peter, as you can see I have changed it to NTP.

Geoff, thanks for making this EPG updater, it's doing a great job.

[adoxa]

I think you could replace:

Code: Select all

                        {
                            wget -q -O - http://localhost:8001/${sref_uri}  &
                            pid=$!
                            sleep $stream_secs
                            kill $pid 
                            sleep 0.1       # hack to prevent very last "{job #} Terminated" msg appearing
                        } >/dev/null 2>&1   # prevent pollution of background job termination messages

with simply:

Code: Select all

                        wget -q -O /dev/null http://localhost:8001/${sref_uri}?duration=$stream_secs

[Grumpy_Geoff]

Thanks, I'll try it out as it looks much neater. I'll see if I still need to do something with the job termination messages.

[Henk]

Hi Grumpy_Geoff,

I did a manual scan for just SBS and added SBS WorldWatch to my V2. and then added this service to the Favorites.

My typical output has changed from:

=======pre SBS WorldWatch scan ==start======
root@beyonwizv2:~# tail -n 28 /home/root/logs/AutoVisit.220503_105058.log
: SBS ONE HD: SBS World News
: ABC NEWS: ABC News Mornings
: Newsmax Tv United States: Newsmax 06h00 ET
: Sky News Extra 3 National: Sky News Extra 3 National
Number of active service names: 5
Active services: GWN7 / SBS ONE HD / ABC NEWS / Newsmax Tv United States / Sky News Extra 3 National
Didn't match on a service name for Newsmax Tv United States, will scan it
Didn't match on a service name for Sky News Extra 3 National, will scan it
Free tuners: 2
Pattern match for existing tuning: 99E:3286|3A0:3202|263:1010|999:9999|999:9999
Service reference: 1:0:19:2E5:263:1010
- Provider/broadcaster's TSID:ONID is 263:1010
- Skipping this serviceref as the box is already chomping on this provider
Service reference: 1:0:16:777:327B:1015
- Provider/broadcaster's TSID:ONID is 27B:1015
- Streaming
Service reference: 1:0:19:3A5:3A0:3202
- Provider/broadcaster's TSID:ONID is 3A0:3202
- Skipping this serviceref as the box is already chomping on this provider
Service reference: 1:0:1:957:99E:3286
- Provider/broadcaster's TSID:ONID is 99E:3286
- Skipping this serviceref as the box is already chomping on this provider
Service reference: 1:0:1:A35:A00:101B
- Provider/broadcaster's TSID:ONID is A00:101B
- Streaming
Finished looping through services to process
Finished cycle: 03/05/22 18:31:23
=======pre SBS WorldWatch scan ==end======

to

=======post SBS WorldWatch scan ==start======
root@beyonwizv2:~# tail -n 28 /home/root/logs/AutoVisit.220509_093847.log
Didn't match on a service name for ABC NEWS, will scan it
Didn't match on a service name for Newsmax Tv United States, will scan it
Free tuners: 5
Pattern match for existing tuning: 999:9999|999:9999|999:9999
Service reference: 1:0:19:3A5:3A0:3202
- Provider/broadcaster's TSID:ONID is 3A0:3202
- Streaming
Finished looping through services to process
Finished cycle: 10/05/22 02:08:11

-----------------
10/05/22 02:28:10
Current recording service name/program: GWN7: An American Affair
: ABC NEWS: DW News
: Newsmax Tv United States: Newsmax 12h00 ET
Number of active service names: 3
Active services: GWN7 / ABC NEWS / Newsmax Tv United States
Didn't match on a service name for GWN7, will scan it
Didn't match on a service name for ABC NEWS, will scan it
Didn't match on a service name for Newsmax Tv United States, will scan it
Free tuners: 5
Pattern match for existing tuning: 999:9999|999:9999|999:9999
Service reference: 1:0:19:3A5:3A0:3202
- Provider/broadcaster's TSID:ONID is 3A0:3202
- Streaming
Finished looping through services to process
Finished cycle: 10/05/22 02:28:42

=======post SBS WorldWatch scan ==end======

Any ideas on what is going on here.
SBS WorldWatch does not have any EPG yet.

[adoxa]

I did a manual scan for just SBS and added SBS WorldWatch to my V2. and then added this service to the Favorites.

AutoVisit gets its services from the terrestrial bouquet; if you didn't set "Clear before scan" to "no" then it will only contain SBS.

[Henk]

Hi Adoxa,

Thanks for the insight!