Daily recordings not working.

[prl]

...
Why do all local timers have to be pushed to IceTV?

My understanding is that that was an IceTV requirement. It was also the way the old IceTV server worked with the DP series.

Perhaps a new flag can be added to the timer creation screen that specifies the type of timer being created. The options being "Local" or "IceTV". Only timers tagged as "IceTV" should be synchronised with IceTV. Obviously all timers coming from IceTV should be tagged as such. This change would give users clarity as to the scope and place for any created timers as well as freedom to create timers independently of IceTV.
...

All the mechanisms to prevent IceTV from knowing about locally-created timers and to distinguish locally-generated timers from IceTV timers already exist. IceTV-created timers have an IceTV timer id. At the moment, if a timer is created on the Beyonwiz the IceTV plugin is informed about its creation and the plugin informs the IceTV server, which allocates it an IceTV timer id. If the IceTV server wasn't notified about the timer, it wouldn't get an IceTV timer id assigned to it and it would be distinguishable from an IceTV timer. AutoTimer-generated timers have the attribute isAutoTimer set to True.

[IanSav]

Hi Prl,

I have seen and guessed much of the operation. My point is that the IceTV flag is not exposed to users so they have no option to separate or integrate local timers with IceTV. Amongst other things, IceTV originally mandated padding times to be used but Beyonwiz no longer forces that directive. The timer synchronisation option should also be handed over to user control.

Regards,
Ian.

[Grumpy_Geoff]

This problem of repeating timers being deleted appears to have been caused by the changes in firmware 20170310 (also see here).

This fix "Locally created timers not handled properly in IceTV" wouldn't have been visible to non-beta punters until 20170310 right?
Hence the statement from sluggahhh that this issue arose from the recent upgrade to 20170310.
So, did the Timer Monster get released as a side-effect of the fix?

[Grumpy_Geoff]

This weekday "ABC News At Noon" timer disappeared -
name="ABC News At Noon"
repeated="31"
ice_timer_id="16285299137673692582"

It was listed at boot (~10:40) -

Code: Select all

{532}<    50.112> [RecordTimer] Record RecordTimerEntry(name=ABC News At Noon, begin=Fri Apr 14 11:55:00 2017, end=Fri Apr 14 12:35:00 2017, serviceref=1:0:19:2E5:261:3201:EEEE0000:0:0:0:, justplay=0, isAutoTimer=0, ice_timer_id=16285299137673692582)

There was an IceTV update at 17:12:03 ("last_update_time":"1492161123") listing the "channels", and the "shows", and the "shows"entry contained the delete ("forget") for the "ABC News At Noon" timer -

Code: Select all

"shows":[],"timers":[{"id":"16285299137673692582","device_id":"176230","channel_id":"2687","name":"ABC News At Noon","show_id":"131424503","start_time":"2017-04-14T04:05:00+00:00","duration_minutes":"6","action":"forget","state":"waiting","message":"Unschedule requested via PIMP web interface.","series_recording_id":"0","keyword_id":"0"}]}

Then this entry, but I don't know which timer this refers, but I'm guessing it's the "ABC News At Noon" timer because if you add the 20-seconds prepare time to 17:11:43 you get the (then) current time -

Code: Select all

{651}< 23455.706> [RecordTimer] record time changed, start prepare is now: Fri Apr 14 17:11:43 2017

Then the confirmation of the delete going back to IceTV -

Code: Select all

{617}< 23456.296> [IceTV] DELETE http://api.icetv.com.au/shows/timers/16285299137673692582?email_address={email address}&token={token}&application_version=20161209

What I noticed that was strange was the duration of the program -
"duration_minutes":"6"
I checked the previous log for when it was created, and the POST message had the 6-min duration (snippet) -

Code: Select all

{9434}<  8946.496> [IceTV] {"name": "ABC News At Noon", "start_time": "2017-04-14T04:05:00+00:00", "duration_minutes": 6, "channel_id": 2687, "state": "pending", "action": "record", "message": "Created by Beyonwiz T4", "device_id": 176230}

But the timer was for 40 minutes, as shown in the log entries a few lines prior -

Code: Select all

{9389}<  8945.812> [RecordTimer] Record RecordTimerEntry(name=ABC News At Noon, begin=Fri Apr 14 11:55:00 2017, end=Fri Apr 14 12:35:00 2017, serviceref=1:0:19:2E5:261:3201:EEEE0000:0:0:0:, justplay=False, isAutoTimer=False)

Is IceTV deleting these timers because they (maybe sometimes?) have a dodgy duration?

Over to you, Mr Code Meister

[Grumpy_Geoff]

I found another error - it may be related. The repeat timer effectively changes to a single timer.

I set a repeat for "Grandpa In My Pocket". EPG entry was for 13:35/13:45, and I (initially) set start/end of 13:30/14:50 (I meant to set 13:50).

Code: Select all

{532}< 94616.099> [RecordTimer] record time changed, start prepare is now: Sat Apr 15 13:29:40 2017
{532}< 94616.100> [RecordTimer] Record RecordTimerEntry(name=Grandpa In My Pocket, begin=Sat Apr 15 13:30:00 2017, end=Sat Apr 15 14:50:00 2017, serviceref=1:0:1:2E2:261:3201:EEEE0000:0:0:0:, justplay=False, isAutoTimer=False)
{532}< 94616.516> KEY: 399 break KEY_GREEN ('GREEN',)

IceTV got told of the timer, but with a strange start time and duration !!! Duration of 46 mins I'm guessing is an event time derived as 80 mins less my standard pre/post-padding of 10/24 (i.e. 80 - 34 = 46).

Code: Select all

{26008}< 94616.731> [IceTV] {"name": "Grandpa In My Pocket", "start_time": "2017-04-15T05:40:00+00:00", "duration_minutes": 46, "channel_id": 37, "state": "pending", "action": "record", "message": "Created by Beyonwiz T4", "device_id": 176230}

Is the start time supposed to indicate the half-way point of the event?

IceTV responds with - "id":"16285583978596140971"

I then altered the end time to 13:50 (making the timer 13:30/13:50)
IceTV gets that change, still with the 'strange' start time, but now with a negative duration -

Code: Select all

{617}< 94644.842> [IceTV] {"timers": [{"start_time": "2017-04-15T05:40:00+00:00", "message": "Will record on Beyonwiz T4", "id": "16285583978596140971", "duration_minutes": -14, "state": "pending"}]}

After the recording had already started, at 13:42 ('Date': 'Sat, 15 Apr 2017 05:42:03 GMT') IceTV sends a timer add and delete, effectively changing the record times and removing the repeat.

Code: Select all

"timers":[
           { "id":"16285584911207826556","device_id":"176230","channel_id":"37","name":"Grandpa In My   Pocket","show_id":"131493655","start_time":"2017-04-15T05:35:00+00:00","duration_minutes":"10","action":"record","state":"waiting","message":"Queued for delivery to device","series_recording_id":"0","keyword_id":"0"
           },
           {"id":"16285583978596140971","device_id":"176230","channel_id":"37","name":"","show_id":"0","start_time":"2017-04-15T05:40:00+00:00","duration_minutes":"46","action":"forget","state":"waiting","message":"Unschedule requested via PIMP web interface.","series_recording_id":"0","keyword_id":"0"
           }
         ]

The T4 then creates the new timer, with my standard padding -

Code: Select all

{651}< 97256.617> [RecordTimer] Record RecordTimerEntry(name=Grandpa In My Pocket, begin=Sat Apr 15 13:25:00 2017, end=Sat Apr 15 14:09:00 2017, serviceref=1:0:1:2E2:261:3201:EEEE0000:0:0:0:, justplay=False, isAutoTimer=False, ice_timer_id=16285584911207826556)

I've previously mentioned this timer repeat -> single change. This is the 3rd time I've seen it in my testing for this thread. This time we've got some debug info - hopefully it's enough to identify the recalcitrant code.

[prl]

The IceTV plugin code always strips off the padding time when it sends a local timer's data to the server. It has no way of knowing whether the timer was in fact padded or not.

If you set a local timer that is shorter than the total of pre- and post-padding, strange things can happen on the IceTV server side when that timer is sent to IceTV from the Beyonwiz.

I'm going to be trying to get the picon updates out by tomorrow, so I probably won't have a proper look at what you've posted until after that, but it looks helpful.

[peteru]

Peteru has access to earlier unpublished specs that I don't have access to, and that I understand he's not allowed to divulge.

Unfortunately I do not. The initial specification was available as a wiki on a third party server. When IceTV went into administration, that resource has vanished without a warning.

It's very likely you have access to a more complete and up to date specification than what I used for the initial implementation. However, there are many issues with the API design, that I don't have time to go into.

[prl]

OK, thanks for the clarification.

[MrQuade]

Overwhelmingly, I record programs by name. But, every so often I record time blocks, pretty much as slugahh described.

This is not possible with IceTV or AutoTImers because these rely on matching text. A repeating timer is ideal for thìs function.

While an Ice fix is in progress, have you tried setting an autotimer with a long custom offset to add a block of padding to the end of the timer?

In sluggah's post, he describes recording the News, then Today Tonight.

In his instance he could simply set a weekday News recording, and then program a custom offset of 45 minutes of post-padding so that Today Tonight is captured as well.

I consider Autotimers as the optimal solution to this issue seeing as IceTV will not attempt to enforce its will on those timers at all. Though it would be good to have the ability to set a "dumb" repeating timer that has a "ignored by Ice" flag on it too.

[Grumpy_Geoff]

Overwhelmingly, I record programs by name. But, every so often I record time blocks, pretty much as slugahh described.

This is not possible with IceTV or AutoTImers because these rely on matching text. A repeating timer is ideal for thìs function.

While an Ice fix is in progress, have you tried setting an autotimer with a long custom offset to add a block of padding to the end of the timer?

In sluggah's post, he describes recording the News, then Today Tonight.

In his instance he could simply set a weekday News recording, and then program a custom offset of 45 minutes of post-padding so that Today Tonight is captured as well.

I consider Autotimers as the optimal solution to this issue seeing as IceTV will not attempt to enforce its will on those timers at all. Though it would be good to have the ability to set a "dumb" repeating timer that has a "ignored by Ice" flag on it too.

I don't think you can do true 'weekday' timers with AutoTimers as there's no day selection that I'm aware of.
A 5-timer generation limit (Record a maximum of x times), resetting on a Monday might be a possible way of achieving the same.
Acknowledging that slugahhh doesn't want to use an AutoTimer as he doesn't want the generated timers 'littering' the timer list -

... IceTv can be used to record both of these shows however it make a mess of your schedule on the Wiz populating a full weeks worth of these two shows straight away.

A work-around, for both slugahhh and Paul_oz53 is to -

• disable IceTV
• create the repeat timer(s)
• re-enable IceTV

A bit of extra effort, but (i) Ice doesn't get to know of the timer, and (ii) it doesn't "make a mess of your schedule on the Wiz"

[MrQuade]

I don't think you can do true 'weekday' timers with AutoTimers as there's no day selection that I'm aware of.

You just go into the filters and select filter "on Weekday" and Include "Weekday". You can also select Weekend, or any combination of days.

[Grumpy_Geoff]

I don't think you can do true 'weekday' timers with AutoTimers as there's no day selection that I'm aware of.

You just go into the filters and select filter "on Weekday" and Include "Weekday". You can also select Weekend, or any combination of days.

Ahh numpty me , I've even posted about that on the forum - http://www.beyonwiz.com.au/forum/viewto ... 20#p135216

[Paul_oz53]

Overwhelmingly, I record programs by name. But, every so often I record time blocks, pretty much as slugahh described.

This is not possible with IceTV or AutoTImers because these rely on matching text. A repeating timer is ideal for thìs function.

While an Ice fix is in progress, have you tried setting an autotimer with a long custom offset to add a block of padding to the end of the timer?

In sluggah's post, he describes recording the News, then Today Tonight.

In his instance he could simply set a weekday News recording, and then program a custom offset of 45 minutes of post-padding so that Today Tonight is captured as well.

I consider Autotimers as the optimal solution to this issue seeing as IceTV will not attempt to enforce its will on those timers at all. Though it would be good to have the ability to set a "dumb" repeating timer that has a "ignored by Ice" flag on it too.

In what I'm about to say, let's not get too serious - this is something I do rarely, usually with sporting events subject to potential weather delays.

Rather than record by the program name, I just record a large overlapping time block - say from 2:00am to 9:00am. I completely ignore program names because that has caused me to miss some action in the past (usually the finish!). The problem with autotimers and IceTV is the same - they are both obsessed with matching text and won't operate if the named text is not there. This can happen if the original event is delayed and some other random program is on at 5:00am.

Usually I use a single timer so evidently not an issue. But some events are on multiple days so a repeating timer is appropriate. Perversely, when the TiVo EPG service dies, this will likely be the only way I can get use from it (barring a miracle resurrection by the TiVo enthusiasts).

I hope prl can find a fix - if anyone can it will be our Wizard God. All hail the Wizard God!

A work-around, for both slugahhh and Paul_oz53 is to -
disable IceTV
create the repeat timer(s)
re-enable IceTV
A bit of extra effort, but (i) Ice doesn't get to know of the timer, and (ii) it doesn't "make a mess of your schedule on the Wiz"

A very handy tip to know. Thanks Grumpy_Geoff.

[slugahhh]

Thanks Geoff will give that a try.

[MrQuade]

Rather than record by the program name, I just record a large overlapping time block - say from 2:00am to 9:00am. I completely ignore program names because that has caused me to miss some action in the past (usually the finish!). The problem with autotimers and IceTV is the same - they are both obsessed with matching text and won't operate if the named text is not there. This can happen if the original event is delayed and some other random program is on at 5:00am.

Unfortunately both IceTVand Autotimers are fundamentally about event matching, so they need some sort of anchor with which to operate from. If you want to be able to set purely time based timers, they'd have to be done outside of either of those two mechanisms.

The more I think about this, the more I think that any timer that is set as a repeat should just be ignored by Ice in much the same way as an Autotimer is.

The way I am seeing this, the only reason that this used to work in the 4.4 firmware was due to a bug in the IceTV plugin's event ID matching code that allowed any timer set from the Beyonwiz interface to remain unmolested since Ice was not able to associate the timer properly with an event and IceTV's servers would consequently not be able to adjust the timer once it was set.

Usually I use a single timer so evidently not an issue. But some events are on multiple days so a repeating timer is appropriate. Perversely, when the TiVo EPG service dies, this will likely be the only way I can get use from it (barring a miracle resurrection by the TiVo enthusiasts).

I suspect you'd probably find that your single timers might also be affected in that their run lengths run the risk of being adjusted by IceTV as well.

I suspect that users like Tezza are also seeing some repercussions of the event-id matching bug fix in that some of his custom-set per-event padding options are being quietly overwritten any time IceTV decides to adjust a timer.

I am guessing that prl's fix for repeating timers is going to involve ensuring that each remove-add pair of operations (required when Ice needs to shift a timer) are now going to be processed in the Wiz together as a matched-pair of operations such that any custom properties of the removed timer are being re-inserted into the newly added timer that replaces it. (Based on some info that prl had posted in this other thread.)

Thanks Geoff will give that a try.

I still do think your particular scenario will be better-served in the long run with an Autotimer rather than the suggested Ice-workaround.

[Paul_oz53]

I suspect you'd probably find that your single timers might also be affected in that their run lengths run the risk of being adjusted by IceTV as well.

I suspect that users like Tezza are also seeing some repercussions of the event-id matching bug fix in that some of his custom-set per-event padding options are being quietly overwritten any time IceTV decides to adjust a timer.

Now that I am aware of this IceTV behaviour, I think I may have experienced that shortening effect too. Just didn't realise at the time what was happening.

We record a lot of sport for a friend who is away FIFO at mine sites. These are the type of events that sometimes don't quite turn out as expected.

Makes me also wonder if it could also affect HDMI-IN recordings - not that I'm suggesting it does.

Cheers, Paul

[MrQuade]

Makes me also wonder if it could also affect HDMI-IN recordings - not that I'm suggesting it does.

It shouldn't do, since any HDMI recordings won't have any event associated with it, or even a service that IceTV recognises, so IceTV won't attempt to manage those timers.

[Grumpy_Geoff]

Rather than record by the program name, I just record a large overlapping time block - say from 2:00am to 9:00am. I completely ignore program names because that has caused me to miss some action in the past (usually the finish!). The problem with autotimers and IceTV is the same - they are both obsessed with matching text and won't operate if the named text is not there. This can happen if the original event is delayed and some other random program is on at 5:00am.

Unfortunately both IceTVand Autotimers are fundamentally about event matching, so they need some sort of anchor with which to operate from. If you want to be able to set purely time based timers, they'd have to be done outside of either of those two mechanisms.

Yes, for purely time-based timers, I create them whilst I have IceTV disabled.

The more I think about this, the more I think that any timer that is set as a repeat should just be ignored by Ice in much the same way as an Autotimer is.

I think it'd be good to still follow scheduling adjustments though.

The way I am seeing this, the only reason that this used to work in the 4.4 firmware was due to a bug in the IceTV plugin's event ID matching code that allowed any timer set from the Beyonwiz interface to remain unmolested since Ice was not able to associate the timer properly with an event and IceTV's servers would consequently not be able to adjust the timer once it was set.

Yes, I posted about that here, I believe fixing issue #512 has now allowed this behaviour.

Usually I use a single timer so evidently not an issue. But some events are on multiple days so a repeating timer is appropriate. Perversely, when the TiVo EPG service dies, this will likely be the only way I can get use from it (barring a miracle resurrection by the TiVo enthusiasts).

I suspect you'd probably find that your single timers might also be affected in that their run lengths run the risk of being adjusted by IceTV as well.

Yes, I posted here of an example that IceTV does change those single timers - How to prevent IceTV updating a manual timer?.
I fixed those by creating them whilst I had IceTV disabled.

I suspect that users like Tezza are also seeing some repercussions of the event-id matching bug fix in that some of his custom-set per-event padding options are being quietly overwritten any time IceTV decides to adjust a timer.

Yes, and possibly the recording location too

I am guessing that prl's fix for repeating timers is going to involve ensuring that each remove-add pair of operations (required when Ice needs to shift a timer) are now going to be processed in the Wiz together as a matched-pair of operations such that any custom properties of the removed timer are being re-inserted into the newly added timer that replaces it. (Based on some info that prl had posted in this other thread.)

This post has an example of Ice adding the replacement of what originally was a repeat, and deleting the repeat timer.
Yes, If prl's fix can cherry-pick from the to-be-deleted repeat and apply that to the usurping timer then that would be ideal, allowing scheduling changes to be followed.

[MrQuade]

The more I think about this, the more I think that any timer that is set as a repeat should just be ignored by Ice in much the same way as an Autotimer is.

I think it'd be good to still follow scheduling adjustments though.

This thing is, if you are concerned about following schedule changes, it means that there is an event to track, in which case the Autotimer can easily be used. A simple time-block timer would not require any of the event schedule matching abilities.

I will wait for prl's response for any other comments, but I am not sure exaclty how he would handle Ice managing extended duration recordings. Since the padding setting is not saved, there would have to be a lot of guesswork on the part of the IceTV plugin to figure out how to retain the duration of the recording in when comparing the old and newly updated timer from Ice. The other settings like repeat and destination that you mention would be fairly easy to figure out how to retain.

[prl]

...
I don't think you can do true 'weekday' timers with AutoTimers as there's no day selection that I'm aware of.
...

There is, but it's a bit hidden.

From the EPG, create an AutoTimer using BLUE Add Auto Timer. Leave the "Only on Weekday" setting enabled, that will set up one day for you which helps confirm that you've found the right place. It's also a hint that selection by day of the week is possible

Then OK to go to the Edit Auto Timer screen.

Scroll down to the second-last entry in Edit Auto Timer, Restriction to certain days of the week. It should show enabled, but that will just be for the single day you chose.

YELLOW Edit filters. That opens the filter editor. Scroll down to Filter and use LEFT/RIGHT to select the "on Weekday" filter.

Is should show a single Include entry for a single day. Scroll down to the entry and use LEFT/RIGHT to select Weekday.

GREEN Save to set the new inclusion and return to the Edit Auto Timer screen. Set the custom padding in "Custom offset" (why can't people use common terminology! ) as discussed previously. Make any other changes. GREEN Save to create the AutoTimer.

[prl]

...
I will wait for prl's response for any other comments

Already done

but I am not sure exaclty how he would handle Ice managing extended duration recordings. ...

I wouldn't. I'd just set two IceTV recordings. Disk space is cheap.

You can change the duration of an IceTV timer, but it's a fragile thing. You have to do it for every instance of the timer (i.e. for every episode), and if IceTV decides to update the timer, for any reason, even if it doesn't change the start or end times, your carefully set start and end will be overwritten using the standard padding.

IIRC tezza used to do this. I wouldn't have the patience. It would end badly.

[MrQuade]

Already done

Not sure if follow you there. What is done?

I was referring to the possibility of having iceTV completely ignore all repeating timers.

[prl]

Already done

Not sure if follow you there. What is done?

I was referring to the possibility of having iceTV completely ignore all repeating timers.

I was replying to what I thought was a comment about me replying to Geoff.

[MrQuade]

I was replying to what I thought was a comment about me replying to Geoff.

No, we'd already discussed the weekday recording thing several posts back.

I was referring this this post, and the following comments.

[Grumpy_Geoff]

Here's my thoughts on potential processing changes.
I am basing this on slugahhh's use case - to not populate the timer list with waiting timers; hence the use of a repeat timer on a named event and not using IceTV nor AutoTimers.
Having a repeat follow an IceTV schedule change is a desired outcome.

If this is considered too difficult (or prl doesn't feel inclined to code it ), then let's have the plugin changed to simply not push the repeat timer to IceTV at all.

1. From the IceTV "timers" message, process delete ("forget") entries first
2. Locate the existing PVR timer, matching "ice_timer_id" value with "id" value from the IceTV delete entry
3. If the delete entry is for a repeat timer -
   1. Grab the "name" from the existing timer and find the corresponding add ("record") entry with the same name and with a "start_time" between timer "begin" and "end"
      ["show_id" is/can be null in the delete entry so we can't use it to directly match the add ("record") entry]
   2. If the matching add entry is not located, then ignore this delete entry (leaving the repeat timer alone)
   3. From the located matching add entry, use "start_time" and "duration_minutes" to update existing timer's "begin" and "end".
      If possible, use the "eit" value from the existing timer to locate the EPG event's times and deduce the repeat's pre/post-padding values that were used, and apply these to the time/duration from the add entry.
      If this is not possible, then use standard padding
      If it is thought re-applying the original padding will result in endless updates from the IceTV server due to programming mismatches, then forget about coding up using the repeat's original padding and just use standard padding instead.
   4. The add entry has now effectively updated the existing timer - no further processing of these 2 add/delete entries is required
4. If the delete entry is not for a repeat, then process as per existing
5. Process add ("record") entries next, ignoring those paired above

If the user doesn't want (the potential of) standard padding to be applied to these repeats on an Ice schedule update, then use AutoTimers. If the user doesn't want AutoTimers to schedule a week's worth (e.g. as per slugahhh "IceTv can be used to record both of these shows however it make a mess of your schedule on the Wiz populating a full weeks worth of these two shows straight away.", then adjust the AutoTimer's "only add timers for next x days" value.

Thoughts?


Cheers,
Geoff

[IanSav]

Hi,

The more I read about issues with working out padding in timers the more I think it becoming more efficient to simply store the start and end padding times used in that timer. It should simply be two items of data added to the timer data array that is unused by everything that currently exists but can be used by new code when going forward. By this I mean that the recordings should still start and end based on the start and end times that are calculated based on the event duration plus the start and end padding. Anything that needs to know about or match an event based on the even start and/or end times can then be enhanced to use the padding data stored in the timer to calculate the required times rather than having a guess.

EDIT: Correct spelling and grammar.

Regards,
Ian.

[MrQuade]

The more I read about issues with working out padding in timers the more I think it becoming more efficvient to simply store the start and end padding times used in that timer.

Yep. If it can be stored in the timer as some extra tags and not upset any existing code, then that sounds like a good idea.

[prl]

The more I read about issues with working out padding in timers the more I think it becoming more efficvient to simply store the start and end padding times used in that timer.

Yep. If it can be stored in the timer as some extra tags and not upset any existing code, then that sounds like a good idea.

I've looked at implementing it, and while it's an attractive idea, there are issues with compatibility with plugins. If it were easy I would have done it, because it is a good idea. Not knowing the padding is the underlying problem in bugs #449 and #450, and in general with the timer interface to the server in the IceTV plugin.

It would be good to have it in the AutoTimer plugin, for example, because it would make the "Guess existing timer based on begin/end" option have a reasonable chance of working properly with the typical padding needed to cope with Australian broadcasting schedule anarchy. But to have that, it would be necessary to have the same changes implemented by all AutoTimer users, or to at least have a liberal sprinkling of hasattr() calls throughout the code.

[prl]

Here's my thoughts on potential processing changes.
I am basing this on slugahhh's use case - to not populate the timer list with waiting timers; hence the use of a repeat timer on a named event and not using IceTV nor AutoTimers.
Having a repeat follow an IceTV schedule change is a desired outcome.

If this is considered too difficult (or prl doesn't feel inclined to code it ), then let's have the plugin changed to simply not push the repeat timer to IceTV at all.

My current inclination is to not push repeat timers to IceTV. There is a slight complication to that, though: if a single timer becomes a repeat timer, its IceTV server side entry needs to be deleted; and conversely, if a repeat timer becomes a one-off timer, it needs to be sent to IceTV.

I still haven't looked at the log entries that Grumpy_Geoff posted in any detail. The key thing that I want to try to find is why (and hence when) the IceTV server is deleting Beyonwiz-set timers. I assume that it's doing it to all timers set on the Beyonwiz side because it doesn't know that the timer is a repeat timer.

1. From the IceTV "timers" message, process delete ("forget") entries first
2. Locate the existing PVR timer, matching "ice_timer_id" value with "id" value from the IceTV delete entry
3. If the delete entry is for a repeat timer -
   1. Grab the "name" from the existing timer and find the corresponding add ("record") entry with the same name and with a "start_time" between timer "begin" and "end"
      ["show_id" is/can be null in the delete entry so we can't use it to directly match the add ("record") entry]
   2. If the matching add entry is not located, then ignore this delete entry (leaving the repeat timer alone)
   3. From the located matching add entry, use "start_time" and "duration_minutes" to update existing timer's "begin" and "end".
      If possible, use the "eit" value from the existing timer to locate the EPG event's times and deduce the repeat's pre/post-padding values that were used, and apply these to the time/duration from the add entry.
      If this is not possible, then use standard padding
      If it is thought re-applying the original padding will result in endless updates from the IceTV server due to programming mismatches, then forget about coding up using the repeat's original padding and just use standard padding instead.
   4. The add entry has now effectively updated the existing timer - no further processing of these 2 add/delete entries is required
4. If the delete entry is not for a repeat, then process as per existing
5. Process add ("record") entries next, ignoring those paired above

The suggested algorithm seems to be based on the assumption that an IceTV timer update is a forget/record pair. That isn't the case, and it's probably just as well, because the IceTV server doesn't always order these entries as forget-then-record (see Timers not deleted before new timers are added). IceTV "record" entries are used for both new timers and updates. If a timer with the supplied IceTV timer id doesn't exist, the "record" entry is used to create a new timer. If a timer with that id exists, the existing timer is updated from the "record" entry.

Using the timer name for matching sounds like a source of unpredictable (though probably infrequent) errors.

An alternative that I've been thinking of is for a repeat timer to delete itself and re-create itself with its next firing time on the IceTV side each time it completes. There are potential problems with this, too, because a timer doesn't necessarily complete (e.g. if the PVR was turned off at the mains when the timer was supposed to run), but I think it may be a reasonably clean approach.

[MrQuade]

I still haven't looked at the log entries that Grumpy_Geoff posted in any detail. The key thing that I want to try to find is why (and hence when) the IceTV server is deleting Beyonwiz-set timers. I assume that it's doing it to all timers set on the Beyonwiz side because it doesn't know that the timer is a repeat timer.

I'm not sure Ice is actually deleting the timers as such though is it? It's just that the next repeat is never getting set.

Wouldn't it be happening any time there is an IceTV server-side change made to an existing repeating timer?
eg.
Say there is an existing repeat timer set on the Beyonwiz which has been pushed up to the Ice servers. If there is a schedule/description/whatever change, Ice would send a remove timer command, then a set timer command, and the repeat information will be getting stripped out. Then when the timer actually fires, there and it is no longer a repeat, then the next timer is never created.

[prl]

I still haven't looked at the log entries that Grumpy_Geoff posted in any detail. The key thing that I want to try to find is why (and hence when) the IceTV server is deleting Beyonwiz-set timers. I assume that it's doing it to all timers set on the Beyonwiz side because it doesn't know that the timer is a repeat timer.

I'm not sure Ice is actually deleting the timers as such though is it? It's just that the next repeat is never getting set.

That's not what Grumpy_Geoffs posted log entry says:

This weekday "ABC News At Noon" timer disappeared -
name="ABC News At Noon"
repeated="31"
ice_timer_id="16285299137673692582"

It was listed at boot (~10:40) -

Code: Select all

{532}<    50.112> [RecordTimer] Record RecordTimerEntry(name=ABC News At Noon, begin=Fri Apr 14 11:55:00 2017, end=Fri Apr 14 12:35:00 2017, serviceref=1:0:19:2E5:261:3201:EEEE0000:0:0:0:, justplay=0, isAutoTimer=0, ice_timer_id=16285299137673692582)

There was an IceTV update at 17:12:03 ("last_update_time":"1492161123") listing the "channels", and the "shows", and the "shows"entry contained the delete ("forget") for the "ABC News At Noon" timer -

Code: Select all

{
    ...
    "shows":[],
    "timers": [
	{
	    "id":"16285299137673692582",
	    "device_id":"176230",
	    "channel_id":"2687",
	    "name":"ABC News At Noon",
	    "show_id":"131424503",
	    "start_time":"2017-04-14T04:05:00+00:00",
	    "duration_minutes":"6",
	    "action":"forget",                        <<<<<<<<< This
	    "state":"waiting",
	    "message":"Unschedule requested via PIMP web interface.",
	    "series_recording_id":"0",
	    "keyword_id":"0"
	}
    ]
}

I've added whitespace to the original post to make it readable.

The IceTV server is sending a delete ("forget") request to the plugin for the timer.

Wouldn't it be happening any time there is an IceTV server-side change made to an existing repeating timer?
eg.
Say there is an existing repeat timer set on the Beyonwiz which has been pushed up to the Ice servers. If there is a schedule/description/whatever change, Ice would send a remove timer command, then a set timer command, and the repeat information will be getting stripped out. Then when the timer actually fires, there and it is no longer a repeat, then the next timer is never created.

In my previous post I said:

[GrumpyGeoff's] suggested algorithm seems to be based on the assumption that an IceTV timer update is a forget/record pair. That isn't the case ... . If a timer with the supplied IceTV timer id doesn't exist, the "record" entry is used to create a new timer. If a timer with that id exists, the existing timer is updated from the "record" entry.

If a normal IceTV timer's times are changed, then no, IceTV doesn't delete the timer and ask for it to be re-created. It just sends a new "record" entry. If a timer with that id already exists, it's updated, and if it doesn't already exist, it's created. If it's updated from the IceTV "record" item, the timer's repeat status is not altered. See Plugins.System.IceTV.plugin.EPGFetcher.processTimers() and ...updateTimer(). The IceTV API documentation says that client-set timers get updated. It doesn't say how, but I can't see why the process would be different (but that may be an incorrect assumption).

It's also not trying to update the timer in GrumpyGeoff's posted log snippet. It's just deleting the timer. There's no "record" to update or re-create it.

Please, everyone, have a look at the IceTV Timer Workflow. It even says what's supposed to happen with client-created timers. It neglects to mention anything about IceTV deleting client-set timers.

[MrQuade]

It's also not trying to update the timer in GrumpyGeoff's posted log snippet. It's just deleting the timer. There's no "record" to update or re-create it.

Apologies, I hadn't gone through the process and realised that the existing timer just gets updated.

Question though. If a repeat timer fires and completes, and then sets a new timer for the next scheduled time, does the new timer inherit the same Ice timer ID? If so, would Ice not be obliged to forget it since it thinks that ID has already fired?

[prl]

This weekday "ABC News At Noon" timer disappeared -
name="ABC News At Noon"
repeated="31"
ice_timer_id="16285299137673692582"

It was listed at boot (~10:40) -

Code: Select all

{532}<    50.112> [RecordTimer] Record RecordTimerEntry(name=ABC News At Noon, begin=Fri Apr 14 11:55:00 2017, end=Fri Apr 14 12:35:00 2017, serviceref=1:0:19:2E5:261:3201:EEEE0000:0:0:0:, justplay=0, isAutoTimer=0, ice_timer_id=16285299137673692582)

There was an IceTV update at 17:12:03 ("last_update_time":"1492161123") listing the "channels", and the "shows", and the "shows"entry contained the delete ("forget") for the "ABC News At Noon" timer -

Code: Select all

"shows":[],"timers":[{"id":"16285299137673692582","device_id":"176230","channel_id":"2687","name":"ABC News At Noon","show_id":"131424503","start_time":"2017-04-14T04:05:00+00:00","duration_minutes":"6","action":"forget","state":"waiting","message":"Unschedule requested via PIMP web interface.","series_recording_id":"0","keyword_id":"0"}]}

Then this entry, but I don't know which timer this refers, but I'm guessing it's the "ABC News At Noon" timer because if you add the 20-seconds prepare time to 17:11:43 you get the (then) current time -

Code: Select all

{651}< 23455.706> [RecordTimer] record time changed, start prepare is now: Fri Apr 14 17:11:43 2017

...

I just want to check a few things. This looks like it's Western Australia time, WAST, GMT+8. Is that right?

The sequence seems to be:
Apr 14 10:40: boot
Apr 14 10:40: timer for ABC News At Noon read from file, recording from Apr 14 11:55 to Apr 14 12:35
Apr 14 11:55: timer starts
Apr 14 12:35: timer ends
Apr 14 17:12: IceTV sends "forget" request for timer

Is that how you read it?

[Grumpy_Geoff]

...
I just want to check a few things. This looks like it's Western Australia time, WAST, GMT+8. Is that right?

The sequence seems to be:
Apr 14 10:40: boot
Apr 14 10:40: timer for ABC News At Noon read from file, recording from Apr 14 11:55 to Apr 14 12:35
Apr 14 11:55: timer starts
Apr 14 12:35: timer ends
Apr 14 17:12: IceTV sends "forget" request for timer

Is that how you read it?

Yes, Perth (as per user details )
Yes, sequence is correct.

I'm guessing IceTV deleted that timer because the start and duration didn't match the Ice server. The times didn't match because I varied them from my standard (10/24 pre/post) padding.
Ice details sent for event from "shows" -

Code: Select all

{"id":"131424503","series_id":"40075","episode_id":"0","channel_id":"2687","date":"0","start":"2017-04-14T04:00:00+00:00","stop":"2017-04-14T04:30:00+00:00","title":"ABC News At Noon","subtitle":"","desc":"The latest from ABC News, following today's top stories & coverage of events as they unfold. Plus comprehensive analysis & original reporting from ABC reporters around Australia & the world.","category":[{"name":"News","eit":"0x20"}],"language":"English","country":"Australia","video":{"aspect":"16:9","colour":"YES","quality":"SDTV"},"subtitles":{"onscreen":"English"},"part_of_series":"Yes","rating":""}

Start time of 12:00 and stop time of 12:30.
Possibly IceTV deleted it because it had 'badly' time-warped/mangled start/duration times when compared with the server. Most likely the "duration_minutes":"6" would have had Ice chucking a wobbly

Here's the timer entry -

Code: Select all

<timer begin="1492142100" end="1492144500" serviceref="1:0:19:2E5:261:3201:EEEE0000:0:0:0:" repeated="31" rename_repeat="1" name="ABC News At Noon" description="The latest from ABC News, following today&apos;s top stories & coverage of events as they unfold. Plus comprehensive analysis & original reporting from ABC reporters around Australia & the world." afterevent="auto" eit="42869" tags="ABC_News_At_Noon" disabled="0" justplay="0" always_zap="0" descramble="1" record_ecm="0" isAutoTimer="0" ice_timer_id="16285299137673692582">
<log code="15" time="1492134531">record time changed, start prepare is now: Fri Apr 14 11:54:40 2017</log>
</timer>

You can see from 'log code 15' that the timer was created 14/4 09:48:51 (AWST)


I've still got 10 repeat timers present. Eight have an ice_timer_id. They're all still chugging away. When I've time, I'll check on their mismatch to the server and report back.

[prl]

...
Yes, Perth (as per user details )
Yes, sequence is correct.

I didn't think to check your user details. I just looked at your sig and who posted the most recent Perth scan data for picons (it was MrQuade).
Thanks for the confirmation.

...
Possibly IceTV deleted it because it had 'badly' time-warped/mangled start/duration times when compared with the server. Most likely the "duration_minutes":"6" would have had Ice chucking a wobbly

Maybe. But why's it only doing it 4 3/2 hours after the timer finished? The update time, ~17:00 WAST doesn't correspond to IceTV's "big daily update", which is usually ~14:00-15:00 AEST on the east coast; I'm not sure whether it's ~14:00-15:00 WAST or ~12:00-13:00 WAST, but neither is anywhere near 17:00 WAST.

Maybe I need to ask Daniel Hall at IceTV.

I've still got 10 repeat timers present. Eight have an ice_timer_id. They're all still chugging away. When I've time, I'll check on their mismatch to the server and report back.

How is it that two don't have an ice_timer_id? Did you create them any differently?

Do any of them have standard padding? That would be a good test of whether the deletion is padding-related.

[IanSav]

Hi Prl,

Please, everyone, have a look at the IceTV Timer Workflow. It even says what's supposed to happen with client-created timers. It neglects to mention anything about IceTV deleting client-set timers.

I had a quick look at the IceTV page. Do you know the logic behind this statement in the section marked "Local Timers":

Timers created locally on the TV Recorder must be sent to the IceTV server, ...

This staement isn't explained or justified in the logic cycle. IceTV does not do any timer conflict processing. Why *must* this happen?

It seems to me if this doesn't happen then everything would work just fine. IceTV does it's thing and the user is free to do their thing and the local PVR can arbitrate and scheduling issues. This also has the added benefit of better protecting the user's viewing privacy.

Even if this has a purpose (like using the data to make viewing suggestions, etc) the user should still be given the option to protect their privacy.

Regards,
Ian.

[prl]

I have no idea what's behind the requirement. It is also the case in the old API used by the DP series.

[Paul_oz53]

Hi Prl,

Please, everyone, have a look at the IceTV Timer Workflow. It even says what's supposed to happen with client-created timers. It neglects to mention anything about IceTV deleting client-set timers.

I had a quick look at the IceTV page. Do you know the logic behind this statement in the section marked "Local Timers":

Timers created locally on the TV Recorder must be sent to the IceTV server, ...

This staement isn't explained or justified in the logic cycle. IceTV does not do any timer conflict processing. Why *must* this happen?

It seems to me if this doesn't happen then everything would work just fine. IceTV does it's thing and the user is free to do their thing and the local PVR can arbitrate and scheduling issues. This also has the added benefit of better protecting the user's viewing privacy.

Even if this has a purpose (like using the data to make viewing suggestions, etc) the user should still be given the option to protect their privacy.

Regards,
Ian.

I agree with the thought here - I'd be content if IceTV only acted on user input via the app (95% of what I do) or the webpage and did not attempt to second guess what I am doing with a manual timer. From what I can gather though, IceTV view user timers in the EPG as a third input source and so IceTV tries to also manage those as named program events. In theory, in a steady-state world and for simple events with 'standard' padding, this should be ok and invisible to the user.

An important assumption here is that the IceTV guide is very accurate and always up to date. My experience would suggest often it is not. Hence, I will occasionally set a time based recording to record a channel in case something special I expect to be on that channel displaces a named program. What matters to me is the timeslot, not the event.

If IceTV sees the event change after I have input the timer I DO NOT want it touched!

But, as this thread points out, IceTV is also incapable of following manual user padding correctly, resulting in failed recordings not capturing the intended follow on from an event. Also, when it captures a repeating timer it ignores the repeat flag and thus kills a repeating timer.

It seems we are debating how to overcome perceived deficiencies in the IceTV approach with minimal input from IceTV. I'm inclined to wonder if the option at least some of us think we need is an option to: Hide manual timers from IceTV - yes/no (default no).

FWIW.
Paul

[Grumpy_Geoff]

...

I've still got 10 repeat timers present. Eight have an ice_timer_id. They're all still chugging away. When I've time, I'll check on their mismatch to the server and report back.

How is it that two don't have an ice_timer_id? Did you create them any differently?

Do any of them have standard padding? That would be a good test of whether the deletion is padding-related.

My padding settings are 10/24 pre/post.

Ten News: 16:55 - 18:05; non-standard padding of 5/5 (event is 17:00 - 18:00)
has ice_timer_id, weekday repeat, fired 3 times so far

Seven News; 17:55 - 19:10; non-standard padding of 5/40 (event is 18:00 - 18:30, I'm also including the following Today Tonight, as per slugahhh's OP)
no ice_timer_id, as timer created as per 'work-around' here, weekday repeat, fired 5 times so far

National Nine News; 17:55 - 19:10; non-standard padding of 5/10 (event is 18:00 - 19:00)
has ice_timer_id, daily repeat fired 6 times so far

ABC News; 18:55 - 19:40; non-standard padding of 5/10 (event is 19:00 - 19:30)
has ice_timer_id, daily repeat fired 8 times so far

Rage; 21:45 - 23:04; standard padding (at time of creation, latest event is 21:55 - 22:30)
has ice_timer_id, daily repeat fired 6 times so far

M*A*S*H"; 10:20 - 11:24; standard padding (event is 10:30 - 11:00)
has ice_timer_id, weekday repeat fired once (timer only created today)

Daniel Tiger's Neighbourhood; 11:15 - 11:30; non-standard padding of 0/0 when created, (event is now 11:10 - 11:25)
no ice_timer_id (created when IceTV was enabled; but not within Ice's acceptable pre-start time), daily repeat fired 8 times so far

Seven Morning News; 11:20 - 12:24; standard padding (event is 11:30 - 12:00)
has ice_timer_id, weekday repeat, fired 4 times so far

Hogan's Heroes; 11:20 - 12:24; standard padding (event is 11:30 - 12:00)
has ice_timer_id, weekday repeat; fired once (timer only created today)

Al Jazeera English News; 18:50 - 19:53; atandard pre-padding, non-standard post-padding of 23 (event is 19:00 - 19:30)
has ice_timer_id, specified days (Sun, Mon, Tue) repeat; fired 3 times on 3 specified days.

[prl]

There doesn't seem to be a clear pattern of odd padding causing the problem, or of why IceTV sometimes decides to kill one off.

Perhaps this is one best handballed to Daniel Hall at IceTV. Along with the question of what is supposed to be done with repeat timers

[prl]

Supplementary question. Do you still have the log with the deletion of the ABC News at Noon timer?

[Grumpy_Geoff]

Supplementary question. Do you still have the log with the deletion of the ABC News at Noon timer?

Yes. What do you want me to look for?

[prl]

For now, nothing, but I'd like to be able to offer a copy to Daniel, if that's OK with you.

[Grumpy_Geoff]

For now, nothing, but I'd like to be able to offer a copy to Daniel, if that's OK with you.

It's 4 days of logging and 14MB in size, but that'll zip down of course.
Dan wouldn't need past the "forget" for that timer though, which is about 1/5th of the way through.
He'd need the previous log file too, as that contains the repeat timer creation and POST up to Ice,
Wouldnt the log sippets of the [IceTV] blocks dealing with ABC News at Noon be enough? Apart from the EPG fetch at startup (5,500 entries), there's only a few dealing with ABC News at Noon event or timer.
I'd be surprised if they haven't still got these interactions audited.

[prl]

I'm not sure how much of it would be useful to IceTV, but if it's not there, then the option isn't open.

I was thinking that having the client side logs might make it easier for them to find the matching logs on the server.

[Grumpy_Geoff]

...
Question though. If a repeat timer fires and completes, and then sets a new timer for the next scheduled time, does the new timer inherit the same Ice timer ID? If so, would Ice not be obliged to forget it since it thinks that ID has already fired?

Yes, the repeat timer has the same ice_timer_id value. The timer is updated with the new begin/times (and also prepare, start and stop recording states along the way).
I think you raise a good point; what does the IceTV server feel about a timer that it believes is past its use by date?

[Grumpy_Geoff]

There's still the other issue of IceTV replacing a repeat timer with a single timer, e.g. as posted here for 'National Nine News' and here for 'Grandpa In My Pocket'. It also happened to a repeat for Rage that I was testing.

Paul_oz53's post here and also here sounds like another example of this.

It looks to me like Paul has padding of 10/30 and this shouldn't have sent dud times to Ice for the early morning or nightly news events.
I accept the possibility that IceTV didn't like my start/duration times for these repeats (due to me altering the times from those that used standard padding), and then issued the "forget" and "record" actions to remove the existing 'dodgy time' repeat and set a new (single) timer. That doesn't match what happened to Paul though.


slugahhh's repeat timer to cover both Seven Nightly News and Today Tonight (here and here) is unlikely to work in ny experience. I've previously posted of Ice altering 'extended duration' timers back to the 1st event's duration.
I believe that case can be achieved by disable IceTV / create repeat (or extended time block) timer / re-enable IceTV.

[Paul_oz53]

Paul_oz53's post here and also here sounds like another example of this.

It looks to me like Paul has padding of 10/30 and this shouldn't have sent dud times to Ice for the early morning or nightly news events.
I accept the possibility that IceTV didn't like my start/duration times for these repeats (due to me altering the times from those that used standard padding), and then issued the "forget" and "record" actions to remove the existing 'dodgy time' repeat and set a new (single) timer. That doesn't match what happened to Paul though.

Good guess - my padding is 10/30 and it correctly highlighted the named programs. Why one failed and one worked until I deleted it is a mystery to me. Both were made with IceTV active.
Paul

[prl]

I've posted a question about how PVR-set repeating timers are supposed to interact with IceTV, and why IceTV sometimes decides to delete them on the IceTV forum in IceTV API documentation.