duplicate SBS timer jobs created by AutoTimer V4.3.2 on V2 with release 19.3

[eppo77]

For the past month or so I keep getting some duplicate timer jobs created by AutoTimer, for SBS ONE HD and SBS VICELAND HD.
SBS recently changed their EPG management software.
I've noticed program descriptions a few days out are incomplete, and those with a start time within the next 24 to 72 hours get updated in the evening. This is also when the extra Timer jobs appear.
I've been in touch with SBS tech support, who tell me EPG entries can't be altered.
I suspect that those EPG entries that seem to get updated actually get deleted, and are then replaced with a new EPG entry, with the same Event name, but a different Event ID. As far as AutoTimer is concerned, this 'new' Event ID then needs to be added as a Timer job. Unfortunately the old Timer job, with the no longer valid Event ID, doesn't get purged automatically.
Most evenings I have to manually delete all SBS related AutoTimer generated Timer jobs, followed by manually triggering AutoTimer to run.
Is there any way of automating the purging of AutoTimer generated Timer jobs with invalid Event IDs?

[Grumpy_Geoff]

Enable AutoTimer setting "Guess existing timer based on begin/end".
From live TV, MENU>>AutoTimers, then MENU>>Settings.

Enabling "Guess existing timer based on begin/end" just allows an existing timer on the same service to be modified to match the new EPG times, if the original event has been replaced by the broadcaster (different EIT) and the time similarity is greater than 80%, rather than creating a new timer.
You can tell when this "guess" has occurred, as the timer's log entries (press i/INFO on the timer entry) will have an additional entry -
"[AutoTimer] Warning, AutoTimer {x} messed with a timer which might not belong to it: {y} ."

Do you use a uniqueness check on these AutoTimer definitions?

[Grumpy_Geoff]

Oh I forgot, you should be running adoxa's "Deswapper" plugin. It will swap around the SBS crap short and extended descriptions.
viewtopic.php?f=47&t=13752&p=199151#p199151

[raymondjpg]

For the past month or so I keep getting some duplicate timer jobs created by AutoTimer, for SBS ONE HD and SBS VICELAND HD.
SBS recently changed their EPG management software.
I've noticed program descriptions a few days out are incomplete, and those with a start time within the next 24 to 72 hours get updated in the evening. This is also when the extra Timer jobs appear.
I've been in touch with SBS tech support, who tell me EPG entries can't be altered.
I suspect that those EPG entries that seem to get updated actually get deleted, and are then replaced with a new EPG entry, with the same Event name, but a different Event ID. As far as AutoTimer is concerned, this 'new' Event ID then needs to be added as a Timer job. Unfortunately the old Timer job, with the no longer valid Event ID, doesn't get purged automatically.
Most evenings I have to manually delete all SBS related AutoTimer generated Timer jobs, followed by manually triggering AutoTimer to run.
Is there any way of automating the purging of AutoTimer generated Timer jobs with invalid Event IDs?

I have seen this too on a U4. Tried everything I know to stop it, unsuccessfully.

[raymondjpg]

Do you use a uniqueness check on these AutoTimer definitions?

I now routinely use Require description to be unique: On same service.

However, I should add that I also routinely schedule timers for both SBS One and SBS HD, on the same autotimer, using the option to allow timers for the same show on different channels.

I've also tried limiting the autotimer to the Favourites bouquet, thinking that somehow there were other channels being picked up by the autotimer, even though I could not see any in the Terrestrial TV or Last Scanned bouquets.

[Grumpy_Geoff]

I've noticed program descriptions a few days out are incomplete, and those with a start time within the next 24 to 72 hours get updated in the evening. This is also when the extra Timer jobs appear.

I see what you mean about the changing short description, see attached result from an EPG search with http://beyonwizv2/api/epgsearch?search= ... rld%20news

[Grumpy_Geoff]

Do you use a uniqueness check on these AutoTimer definitions?

I now routinely use Require description to be unique: On same service.

A uniqueness test of "Title and all descriptions" is buggy in the version of the plugin in the Beyonwiz repository.
For at least the last 10 months I've been running a modified version to fix that bug. It is very rare that I get a duplicate timer generated - usually as a result of the duplicate not covering at least 80% overlap of the original timer.
That code bug was fixed in the OE-Alliance version of the plugin in January. See the pull request #500 and #501 - https://github.com/oe-alliance/enigma2-plugins/pull/500
It is exactly the code I've been running.

So if you've been using "Title and all descriptions", or not enabled "Guess existing timer based on begin/end" for reasons I described earlier, then I can easily imagine duplicate timers will be created.

[Blot]

What do we install to get that?
I have been using start and end times 5-7:30pm to get pawn stars. It works well as most/all appear in this time slot. I was wondering why I had to do this though. I think I have watched every pawn stars now anyway as they are all repeating, just hoping it ticks over to to start.

I was also wondering what gets checked for uniqueness…. current timers, past timers, epg, recordings?

[Grumpy_Geoff]

What do we install to get that?

This change is really only needed if you use "Title and all descriptions". If you use "Title and Short description" then there's no issue.
I'd not advise it unless you are experienced in Wiz command line access.
The basics are to grab the current Beyonwiz version of AutoTimer.py, and change it to match the GitHub commit.
On the Wiz, rename the existing AutoTimer.pyo (optimised code) as an "original" (as you'll need to revert to this version before any online firmware update, if we ever get a new one).
Once that's done copy the modified AutoTimer.py source code (it'll then be optimised to produce the executable AutoTimer.pyo upon GUI startup, or maybe first access).
Does that sound doable for you?

I have been using start and end times 5-7:30pm to get pawn stars. It works well as most/all appear in this time slot. I was wondering why I had to do this though. I think I have watched every pawn stars now anyway as they are all repeating, just hoping it ticks over to to start.

Yep, that's one way to do it if that timeslot has the "new" program event.
But in this case, I see an event that isn't shown in that window but is shown the next morning and afternoon -

Code: Select all

"date": "Fri 10.06.2022", 
"begin": "09:27", 
"end": "10:00", 
"title": "Pawn Stars", 
"shortdesc": "Go full throttle with the Pawn Stars when Rick and Corey race to check out an unrestored BSA bike from WWII. Will this military motorcycle rev their engines or will the deal stall?", 
"longdesc": "", 

A "Title and Short description" uniqueness test, and removal of the start time window should fix that.

I was also wondering what gets checked for uniqueness…. current timers, past timers, epg, recordings?

The EPG event is compared to current and past timers. It can also be compared to recordings if Require description to be unique is set to "Any service/recording".
EPG events and recordings have short and extended descriptions, whereas timers only have one description field.

[Blot]

Thanks.

Yes it sounds doable… I am a computer programmer. I probably will leave it all be though. A bit lazy in my old age. Thanks for the short description tip.

The pawn stars episodes out of the 6pm-ish semi-primetime are actually repeats of the recent past.

[eppo77]

Thanks for the info Grumpy_Geoff.
Can't say I've spent much time in AutoTimer's setup area, as I wasn't consiously aware of it.
I've just been looking at it, and noticed a "Remove not existing events" toggle towards the end, currently disabled.
Would enabling this solve my problem, instead of enabling "Guess existing timer based on begin/end" and trying to side-load a python script file?
linux and python aren't really my area, I'm more at home on/with Windows, VMS and object pascal.

[Blot]

"Guess existing timer based on begin/end" is probably better. That is what the toppy used to do and I only missed the start of one movie in more than 10 years with zero padding at the start.

I suppose removing non existing events might work, but what if it disappears from the epg (like “Morgan” did at just after midnight today…although it was running).

[Grumpy_Geoff]

I've just been looking at it, and noticed a "Remove not existing events" toggle towards the end, currently disabled.
Would enabling this solve my problem, instead of enabling "Guess existing timer based on begin/end" and trying to side-load a python script file?

The "Remove not existing events" setting has, unfortunately, no actual running code (as it was commented out, and I suspect there's a case-mismatch code bug with it too).
You should enable "Guess existing timer based on begin/end".

[Blot]

ahaha, good to know.

[eppo77]

The "Remove not existing events" setting has, unfortunately, no actual running code (as it was commented out, and I suspect there's a case-mismatch code bug with it too).
You should enable "Guess existing timer based on begin/end".

thanks Grumpy_Geoff, good to know

that would explain why I was still getting duplicates after enabling that feature
I assumed it was because I hadn't been able to do a restart yet as I was still recording stuff on the abc

I'll enable the "Guess existing timer based on begin/end" ...

Oh I forgot, you should be running adoxa's "Deswapper" plugin. It will swap around the SBS crap short and extended descriptions.
viewtopic.php?f=47&t=13752&p=199151#p199151

A uniqueness test of "Title and all descriptions" is buggy in the version of the plugin in the Beyonwiz repository.
For at least the last 10 months I've been running a modified version to fix that bug. It is very rare that I get a duplicate timer generated - usually as a result of the duplicate not covering at least 80% overlap of the original timer.
That code bug was fixed in the OE-Alliance version of the plugin in January. See the pull request #500 and #501 - https://github.com/oe-alliance/enigma2-plugins/pull/500
It is exactly the code I've been running.

... and also editing and side-loading those Python script files you referred to, once I've figured out how to do that, that is

[raymondjpg]

So if you've been using "Title and all descriptions", or not enabled "Guess existing timer based on begin/end" for reasons I described earlier, then I can easily imagine duplicate timers will be created.

I am only using "Title and Short description" and I was certainly getting these duplicate timers with SBS programs.

I am also using Deswapper which maybe has complicated things, I have no idea. Unless SBS has changed its policy then the short description field will still be populated with a program rating. You will know, but I could confirm if necessary by disabling the Deswapper plugin.

I did not pay any attention to program descriptions on the duplicate timers, but I doubt there would have been any changes to program ratings to account for the duplication.

I did however observe a very slight start time difference, typically one minute later, on the duplicates. This appears to be similar to the strategy we used back in the day, when in order to record simulcast programs, the second timer would need to be started at least one minute behind the first.

[Grumpy_Geoff]

I did however observe a very slight start time difference, typically one minute later, on the duplicates.

I don't see you mentioning "Guess existing timer based on begin/end".

If you want me to take a look at your troublesome AutoTimer definitions then I'd be happy to.

[raymondjpg]

I did however observe a very slight start time difference, typically one minute later, on the duplicates.

I don't see you mentioning "Guess existing timer based on begin/end".

If you want me to take a look at your troublesome AutoTimer definitions then I'd be happy to.

No need for a look thanks, Water under the bridge.

I will implement the AutoTimer setting "Guess existing timer based on begin/end".

[eppo77]

The basics are to grab the current Beyonwiz version of AutoTimer.py, and change it to match the GitHub commit.
On the Wiz, rename the existing AutoTimer.pyo (optimised code) as an "original" (as you'll need to revert to this version before any online firmware update, if we ever get a new one).
Once that's done copy the modified AutoTimer.py source code (it'll then be optimised to produce the executable AutoTimer.pyo upon GUI startup, or maybe first access).
Does that sound doable for you?

does that mean renaming: BWizV2/usr/lib/enigma2/python/Plugins/Extensions/AutoTimer/AutoTimer.pyo to retain the original?

A uniqueness test of "Title and all descriptions" is buggy in the version of the plugin in the Beyonwiz repository.
For at least the last 10 months I've been running a modified version to fix that bug. It is very rare that I get a duplicate timer generated - usually as a result of the duplicate not covering at least 80% overlap of the original timer.
That code bug was fixed in the OE-Alliance version of the plugin in January. See the pull request #500 and #501 - https://github.com/oe-alliance/enigma2-plugins/pull/500
It is exactly the code I've been running.

AutoTimer.py: https://github.com/oe-alliance/enigma2-plugins/tree/master/autotimer/src

and downloading AutoTimer.py from https://github.com/oe-alliance/enigma2- ... otimer/src
then copying the downloaded AutoTimer.py to BWizV2/usr/lib/enigma2/python/Plugins/Extensions/AutoTimer/
and then restarting the BWizV2


I've got Deswapper_1.1.3_all running, and AutoTimer created SBS timers four or more days into the future now include the truncated program descriptions.
How will this uniqueness test for "Title and all descriptions" cope when the relevant SBS EPG entry is replaced with one where the description is no longer truncated, as and when it's start time gets close enough?

The abc recently had back to back ~<=10 minute episodes of "Nightwalkers".
Would such back to back short programs cause problems if tested for a unique title only (my default padding is set to 5 min pre, and 10 min post)?

[Grumpy_Geoff]

does that mean renaming: BWizV2/usr/lib/enigma2/python/Plugins/Extensions/AutoTimer/AutoTimer.pyo to retain the original?
...
and downloading AutoTimer.py from https://github.com/oe-alliance/enigma2- ... otimer/src
then copying the downloaded AutoTimer.py to BWizV2/usr/lib/enigma2/python/Plugins/Extensions/AutoTimer/
and then restarting the BWizV2

Yeah that's the one, in directory /usr/lib/enigma2/python/Plugins/Extensions/AutoTimer, rename AutoTimer.pyo and copy in the edited AutoTimer.py source code.
There's been other changes to the OE-A code, so I wouldn't be using the whole source file as there may be some dependencies in other files that the Wiz version doesn't yet have. Just replace the code block as identified in the commit and shown below.

Code: Select all

cd /usr/lib/enigma2/python/Plugins/Extensions/AutoTimer
mv AutoTimer.pyo AutoTimer.pyo.orig
cp /hdd/Run_stuff/AutoTimer.py.edited AutoTimer.py
# "GUI should now be restarted"

The code edit should replace this block in the Beyonwiz version (lines 870-880)

Code: Select all

					if foundShort:
						if timer.searchForDuplicateDescription == 2:
							if extdesc1 and extdesc2:
								# Some channels indicate replays in the extended descriptions
								# If the similarity percent is higher then 0.7 it is a very close match
								retValue = (0.7 < SequenceMatcher(lambda x: x == " ", extdesc1, extdesc2).ratio())
						else:
							retValue = True
			else:
				retValue = True
		return retValue

with this block from the OE-A version -

Code: Select all

					if foundShort:
						# At this point we assume the similarity match to be True
						# unless we have been asked to check Extended Descriptions
						# and we have *both* Extended Descriptions in place;
						# in which case we test them.
						retValue = True
						if timer.searchForDuplicateDescription == 2:
							if extdesc1 and extdesc2:
								# Some channels indicate replays in the extended descriptions
								# If the similarity percent is higher then 0.7 it is a very close match
								retValue = (0.7 < SequenceMatcher(lambda x: x == " ", extdesc1, extdesc2).ratio())
			else:
				retValue = True
		return retValue

You'll see the code difference between these two:
The modified OE-Alliance version -
https://github.com/oe-alliance/enigma2- ... er.py#L904
The current Beyonwiz version (copied from the OE-Alliance version at the time firmware 20211010 was created) -
https://github.com/beyonwiz/enigma2-plu ... er.py#L870

Here's the Wiz version as pure text in a browser -
https://raw.githubusercontent.com/beyon ... toTimer.py
... and the OE-A version -
https://raw.githubusercontent.com/oe-al ... toTimer.py

You can directly "fetch" these onto the wiz using the command line -
wget -OAutoTimer.py_new$(date +'%d%b') https://github.com/beyonwiz/enigma2-plu ... toTimer.py
wget -OAutoTimer.py_OE-A_new$(date +'%d%b') https://github.com/oe-alliance/enigma2- ... toTimer.py

Just in case you're not aware and will be editing on a Windows box, use an editor that can be set to Unix-style line endings.

If you want to take the easy approach, just download and copy this already edited version (it has a couple of debug log prints in there).

AutoTimer.py.edited.txt

(32.29 KiB) Downloaded 32 times

Please take a settings backup first. If something goes wrong, you can always delete the plugin and re-install it again.

[Grumpy_Geoff]

I've got Deswapper_1.1.3_all running, and AutoTimer created SBS timers four or more days into the future now include the truncated program descriptions.
How will this uniqueness test for "Title and all descriptions" cope when the relevant SBS EPG entry is replaced with one where the description is no longer truncated, as and when it's start time gets close enough?

I wouldn't have thought you'd need a uniqueness test for "SBS World News" since the (swapped) short description is the same for each event. I'd have thought you'd be using a start-window (e.g. 18:00-19:00).
If SBS are replacing the events, with the replacement having a different EIT, then the enabled "Guess existing timer based on begin/end" setting should mean the existing timer gets updated (assuming the overlap is >= 80%).

The abc recently had back to back ~<=10 minute episodes of "Nightwalkers".
Would such back to back short programs cause problems if tested for a unique title only (my default padding is set to 5 min pre, and 10 min post)?

Were the episodes repeated? If not, then there should be no need for a uniqueness check.
A "Check for uniqueness in" "Title" would only create the one timer. "Title and Short Description" probably would be better, assuming they had unique short descriptions.

[eppo77]

Just in case you're not aware and will be editing on a Windows box, use an editor that can be set to Unix-style line endings.

If you want to take the easy approach, just download and copy this already edited version (it has a couple of debug log prints in there).
AutoTimer.py.edited.txt

Please take a settings backup first. If something goes wrong, you can always delete the plugin and re-install it again.

Thanks for all your help Grumpy_Geoff,
I made the changes this arvo, as I was recording stuff last night.
All I need do to now is update the jobs in my AutoTimer queue to take advantage of the new changes.
I took the lazy option and used your AutoTimer.py.edited.txt file, just to be safe.
I've got Notepad++ which I think is capable of saving files using Unix-style line endings, but then again, maybe not.
I did struggle a bit with file copying using File Commander,
due to it's lack of obvious visual feedback and its noticeable delayed network copy response.
Initially I thought perhaps I lacked write permission to the target folder, but turns out it was just a bit slow,
as though the copy request had been submitted to a batch queue, with a delayed start of several minutes.

I also downgraded the firmware from 19.3_20211010 to 19.3_20210119 prior to those other changes.
I'd upgraded to the later version a week or so ago, following the advice of SBS Support.
An unpleasant undocumented feature I'd previously encountered with that version reared its ugly head again.
Last night, as well as a few nights ago, my V2 kept going into the Power/Restart menu,
ignoring my pushing the Exit key on my remote control most of the time.
And if I did manage to exit out of the Power/Restart menu,
it would just return to the Power/Restart menu after several minutes or more.
I think I eventually managed to stop this cycle by changing channels a few times,
placing it in Standby mode for a few minutes, starting it again, and changing channels a few more times.
On previous occasions I've performed a Reboot, when no Timer jobs were scheduled, to interrupt this undocumented feature.
Oh, and I don't recall seeing any countdown timers when it was behaving like this.

I wouldn't have thought you'd need a uniqueness test for "SBS World News" since the (swapped) short description is the same for each event. I'd have thought you'd be using a start-window (e.g. 18:00-19:00).
If SBS are replacing the events, with the replacement having a different EIT, then the enabled "Guess existing timer based on begin/end" setting should mean the existing timer gets updated (assuming the overlap is >= 80%).

As SBS World News is shown on most evenings, and it's non-truncated program descriptions are usually identical,
it seemed like the perfect example to highlight SBS's new policy of truncating program descriptions,
for programs with a start time four or more days out.

Were the episodes repeated? If not, then there should be no need for a uniqueness check.
A "Check for uniqueness in" "Title" would only create the one timer. "Title and Short Description" probably would be better, assuming they had unique short descriptions.

Sorry, I was being a bit thick.
I realised belatedly that uniqueness testing is an AutoTimer job specific setting, rather than a global setting.

Thanks again for all your help.