WizZilla has arrived

Discussions on Software, File Formats and Conversion.

Moderators: Gully, peteru

prl
Wizard God
Posts: 32712
Joined: Tue Sep 04, 2007 13:49
Location: Canberra; Black Mountain Tower transmitters

Post by prl » Mon Jan 17, 2011 12:10

Peter D wrote:...
I have only tried two or three different recordings all made with 055. I never edit recordings on the BW. I will try some new recordings.
OK, thanks. I'll try to run through some more long transfers, too. Unfortunately, I have to borrow a PC to do it, but I'll try to do it again sometime during the week.
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV

Peter D
Master
Posts: 165
Joined: Mon Apr 07, 2008 10:04
Location: Melbourne

Post by Peter D » Mon Jan 17, 2011 19:38

- reloaded firmware 01.07.350
- recorded channel 7 for 70 minutes
- set up wired connection to PC
- tried Wizzilla to copy recording to PC - FAILED
- tried WixFX to do the same - FAILED

If others do not see this happening with 07 firmware, I suspect the cause is related to the 2TB drive

Is there anyone with a 2TB drive you has tried copying using Wizzilla or WixFX?

IanSav
Uber Wizard
Posts: 16846
Joined: Tue May 29, 2007 15:00
Location: Melbourne, Australia

Post by IanSav » Mon Jan 17, 2011 19:42

Hi Peter D,
Peter D wrote:If others do not see this happening with 07 firmware, I suspect the cause is related to the 2TB drive
How or why are you ruling out a problem with your PC or the network?

Does the recording work properly on the Beyonwiz? Have you tried pushing the recording to your PC with File Player on the Beyonwiz?

I don't think there is enough information to conclude a cause.

Regards,
Ian.

Peter D
Master
Posts: 165
Joined: Mon Apr 07, 2008 10:04
Location: Melbourne

Post by Peter D » Mon Jan 17, 2011 20:33

It used to work fine with the previous DP-P1. I switched to hard wired to rule out any wireless problem.

I have not seen any problem with the original recording when played on the BW.

Copying the recording to the PC when initiated on the BW appears to work fine. Though I have not played the copied file to check.

Peter

IanSav
Uber Wizard
Posts: 16846
Joined: Tue May 29, 2007 15:00
Location: Melbourne, Australia

Post by IanSav » Mon Jan 17, 2011 21:21

Hi Peter,

This makes the problem sound more like a problem on the PC or the client software. Given the number of people using the client software without this issue I think you should concentrate on checking your PC for any possible configuration/firewall/network issues.

Regards,
Ian.

Peter D
Master
Posts: 165
Joined: Mon Apr 07, 2008 10:04
Location: Melbourne

Post by Peter D » Tue Jan 18, 2011 06:15

As I said, nothing has changed execept the BW. Same PC, same PC software, same network.

Several people have had exactly this problem. However, so far I am the only person experiencing it with 01.07 firmware.

prl
Wizard God
Posts: 32712
Joined: Tue Sep 04, 2007 13:49
Location: Canberra; Black Mountain Tower transmitters

Post by prl » Tue Jan 18, 2011 08:07

Hi Ian, Peter D.

As far as I know, the differences between my setup and Peter D's are: I have a DP-Lite (500GB), he has a DP-P2TB (2TB), I'm running WinXP Pro 2002 SP3, Peter D hasn't yet said what version of Windows and/or what additional firewall software he's running, and there are undoubtedly differences between our home networking setups.

Peter D has observed the problem in both 01.07.055, 01.07.069 and 01.07.350. I've been unable to reproduce it in either 01.07.055 or 01.07.350.

The symptoms of the problem that Peter D is experiencing appear to be similar to the problems with WizPnP transfers in 01.05.334, but it's hard to tell without the error message output from getWizPnP.

Peter D, have you tried running getWizPnP directly from the command line? If so, what error message did you get? If not, are you willing to try it out?
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV

Peter D
Master
Posts: 165
Joined: Mon Apr 07, 2008 10:04
Location: Melbourne

Post by Peter D » Tue Jan 18, 2011 08:41

I have run the GetWizPnP command line directly. And, with advanced logging turned on.

I got exactly the same message as the other guy who was doing the testing. I would have to look back in the posts to find it. But it was something like "getWizPnP Failed".

I am running Windows 7 32 bit, with the default Windows firewall.

Peter

Peter D
Master
Posts: 165
Joined: Mon Apr 07, 2008 10:04
Location: Melbourne

Post by Peter D » Tue Jan 18, 2011 08:44

From my earlier post after entering the command line directly:

<file name>.ts/0005: Bad Request
Download failed: Bad Request

where <file name> is the file name being copied.

prl
Wizard God
Posts: 32712
Joined: Tue Sep 04, 2007 13:49
Location: Canberra; Black Mountain Tower transmitters

Post by prl » Tue Jan 18, 2011 08:57

Peter D wrote:From my earlier post after entering the command line directly:

<file name>.ts/0005: Bad Request
Download failed: Bad Request

where <file name> is the file name being copied.
Thanks. Yes, that's the same error as people get with getWizPnP on 01.05.334.

The thing that puzzles me is that a good bit of the recording gets copied before it falls over.

Since I can't reproduce the problem, I really only have two suggestions
  • That I send you a copy of the Perl source for my development version of getWizPnP, which has slightly better error messages now than the getWizPnP in WizZilla. That could give more of a hint as to where the HTTP request to fetch the file is failing, though if it's failing in the same place as it does in 01.05.334, then that's a bit of a dead end. There's a moderate amount of work involved in getting getWizPnP running from the Perl source, in particular you need to install Perl (ActivePerl is known to work) and some Perl library packages. If you're willing to do that you can PM me an email address and I can send you the source. However, even if the cause of the problem can be found, it may not be possible to fix it from the client side.
  • Given the problem also occurs for WizFX, you could take the problem to Beyonwiz Support.
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV

sub3R
Wizard
Posts: 2638
Joined: Sun Sep 09, 2007 12:20
Location: Port Macquarie NSW. Reception from Middle Brother.

Post by sub3R » Sat Jan 29, 2011 14:24

This morning I had another recording that didn?t fully copy from the S1 to the PC?s Ext HDD using WizZilla 1.1.0.60. The S1 was running 01.05.320 f/w & the SBS HD recording ?Coast-3? was being copied in native format. The timer for this recording had been previously renamed using ?clear? to fully clear the name prior to renaming. The recording had been edited using ?keep A-B? & the final recording renamed from ?Coast? to ?Coast-3?. Instead of files 0005 to 0143 (all 32,768kB) being copied, only 0005 to 0141 (all 32,768kB) & 0142 (3,168kB) were copied.

Both 1.0.1.4 & 1.0.2.3 versions of WizFX copied all the recording, as did getWizPnP 0.3.4.0, & the copy command from the S1?s file player.
A second attempt using WizZilla copied the full recording, so this is intermittent on my setup as per my previous failures.
For some unknown reason, both the above versions of WizFX only gave the folder a name of ?Coast-3.tvwiz? instead of ?Coast-3 - Fri Jan 28 2011.tvwiz?.

Copying with WizZilla a second time, I noticed near the end of the copy process, the recording disappeared from WizZilla?s queue window before 0141 had finished copying (observed with Windows Explorer). Files 0141, 0142 & 0143 continued copying successfully after this. (added 3-Feb-11): There is approx an indicated 25sec of download time remaining when the recording disappears from WizZilla?s queue window. (end addition)

I also notice that getWizPnP.exe 0.3.4.0 (as used by WizZilla 1.1.0.60) displays:
?SBS HD: Coast-3/♣Southport to Whitehaven - This stretch of coast is not just the? ?.
Whereas getWizPnP.exe 0.4.3a (latest compiled version) displays:
?SBS HD: Coast-3/Southport to Whitehaven - This stretch of coast is not just the? ?.
The ?♣? has been left out, which is probably irrelevant to the above copy issue.

I have ?--debug=trunc? information for this recording if needed.
Last edited by sub3R on Thu Feb 03, 2011 15:17, edited 1 time in total.
Dennis
U4, Bluey USB tuner, WizTV > Yamaha RX-V3900 > Sony KDL46X2000 TV ||
U4, Bluey USB tuner > Sony KD-43X85J TV > Yamaha YAS-209 || FTA EPG ||
Harmony 650s || (U4s on 19.3.20200901 & T2 on 19.3.20200823) ||
Technicolor DJA0230TLS modem/router, Ethernet LAN, Win10 Home 64 ||

prl
Wizard God
Posts: 32712
Joined: Tue Sep 04, 2007 13:49
Location: Canberra; Black Mountain Tower transmitters

Post by prl » Sat Jan 29, 2011 16:31

sub3R wrote:...
I also notice that getWizPnP.exe 0.3.4.0 (as used by WizZilla 1.1.0.60) displays:
?SBS HD: Coast-3/♣Southport to Whitehaven - This stretch of coast is not just the? ?.
Whereas getWizPnP.exe 0.4.3a (latest compiled version) displays:
?SBS HD: Coast-3/Southport to Whitehaven - This stretch of coast is not just the? ?.
The ?♣? has been left out, which is probably irrelevant to the above copy issue.
...
Yep. This was fixed in getWizPnP 0.4.2.
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV

prl
Wizard God
Posts: 32712
Joined: Tue Sep 04, 2007 13:49
Location: Canberra; Black Mountain Tower transmitters

Post by prl » Sat Jan 29, 2011 17:12

sub3R wrote:This morning I had another recording that didn?t fully copy from the S1 to the PC?s Ext HDD using WizZilla 1.1.0.60. The S1 was running 01.05.320 f/w & the SBS HD recording ?Coast-3? was being copied in native format. The timer for this recording had been previously renamed using ?clear? to fully clear the name prior to renaming. The recording had been edited using ?keep A-B? & the final recording renamed from ?Coast? to ?Coast-3?. Instead of files 0005 to 0143 (all 32,768kB) being copied, only 0005 to 0141 (all 32,768kB) & 0142 (3,168kB) were copied.

Both 1.0.1.4 & 1.0.2.3 versions of WizFX copied all the recording, as did getWizPnP 0.3.4.0, & the copy command from the S1?s file player.
A second attempt using WizZilla copied the full recording, so this is intermittent on my setup as per my previous failures.
For some unknown reason, both the above versions of WizFX only gave the folder a name of ?Coast-3.tvwiz? instead of ?Coast-3 - Fri Jan 28 2011.tvwiz?.

Copying with WizZilla a second time, I noticed near the end of the copy process, the recording disappeared from WizZilla?s queue window before 0141 had finished copying (observed with Windows Explorer). Files 0141, 0142 & 0143 continued copying successfully after this.
...
This sounds like WizZilla may not be interacting with getWizPnP correctly. Could you try running the compiled getWizPnP 0.3.4 a few times to see whether it really does do the copy consistently?
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV

sub3R
Wizard
Posts: 2638
Joined: Sun Sep 09, 2007 12:20
Location: Port Macquarie NSW. Reception from Middle Brother.

Post by sub3R » Sat Jan 29, 2011 18:02

prl wrote:Yep. This was fixed in getWizPnP 0.4.2.
Ah yes. I must remember to read your release notes before posting.
prl wrote:This sounds like WizZilla may not be interacting with getWizPnP correctly. Could you try running the compiled getWizPnP 0.3.4 a few times to see whether it really does do the copy consistently?
I?ll hammer that over the next few days, Peter.
Dennis
U4, Bluey USB tuner, WizTV > Yamaha RX-V3900 > Sony KDL46X2000 TV ||
U4, Bluey USB tuner > Sony KD-43X85J TV > Yamaha YAS-209 || FTA EPG ||
Harmony 650s || (U4s on 19.3.20200901 & T2 on 19.3.20200823) ||
Technicolor DJA0230TLS modem/router, Ethernet LAN, Win10 Home 64 ||

sub3R
Wizard
Posts: 2638
Joined: Sun Sep 09, 2007 12:20
Location: Port Macquarie NSW. Reception from Middle Brother.

Post by sub3R » Thu Feb 03, 2011 15:39

sub3R wrote:
prl wrote:This sounds like WizZilla may not be interacting with getWizPnP correctly. Could you try running the compiled getWizPnP 0.3.4 a few times to see whether it really does do the copy consistently?
I?ll hammer that over the next few days, Peter.
I copied that recording 8 times successfully with getWizPnP.exe 0.3.4.0 using the following command (same as WizZilla uses):
getwizpnp.exe -a -H <the S1?s fixed IP address> -p 49152 --BWName "Coast-3 Jan.28.2011_19.28" -O "M:\PVR_copy-test4" -d --resume

Unfortunately this isn?t conclusive because I also copied that recording 8 times successfully with WizZilla. I couldn't make it fail again using WizZilla. :? :x
I also copied 94.6GB of recordings with getWizPnP.exe 0.3.4.0 overnight successfully.
I made a couple of test SBS HD recordings (one clearing & renaming the timer name & one without clearing the timer name before renaming). After recording, both were renamed but I couldn?t make anything fail using WizZilla or getWizPnP.exe 0.3.4.0 (the HDD wasn?t re-indexed).

I didn?t try any multiple queued copies using WizZilla because I know they will fail (one thing for certain).
sub3R wrote:Copying with WizZilla a second time, I noticed near the end of the copy process, the recording disappeared from WizZilla?s queue window before 0141 had finished copying (observed with Windows Explorer). Files 0141, 0142 & 0143 continued copying successfully after this.
There is approx an indicated 25sec of download time remaining when the recording disappears from WizZilla?s queue window. I have added this to my earlier post.

All the above was with the S1 running 01.05.320 f/w.
Dennis
U4, Bluey USB tuner, WizTV > Yamaha RX-V3900 > Sony KDL46X2000 TV ||
U4, Bluey USB tuner > Sony KD-43X85J TV > Yamaha YAS-209 || FTA EPG ||
Harmony 650s || (U4s on 19.3.20200901 & T2 on 19.3.20200823) ||
Technicolor DJA0230TLS modem/router, Ethernet LAN, Win10 Home 64 ||

prl
Wizard God
Posts: 32712
Joined: Tue Sep 04, 2007 13:49
Location: Canberra; Black Mountain Tower transmitters

Post by prl » Thu Feb 03, 2011 16:29

sub3R wrote:...
Unfortunately this isn?t conclusive because I also copied that recording 8 times successfully with WizZilla. I couldn't make it fail again using WizZilla. :? :x
...
Gremlins? :roll:
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV

sub3R
Wizard
Posts: 2638
Joined: Sun Sep 09, 2007 12:20
Location: Port Macquarie NSW. Reception from Middle Brother.

Post by sub3R » Sun Feb 06, 2011 17:43

prl wrote:
sub3R wrote:...
Unfortunately this isn?t conclusive because I also copied that recording 8 times successfully with WizZilla. I couldn't make it fail again using WizZilla. :? :x
...
Gremlins? :roll:
Yes & still lurking ready to pounce.

Using getWizPnP.exe 0.3.4.0, I have copied another 250GB of recordings successfully - 20 recordings at a time. A few recordings have also been copied successfully with WizZilla but the last recording terminated early. However this same recording copied successfully with getWizPnP.exe 0.3.4.0 (with WizZilla, file 0081 was only 24,416kB & no 0082, whereas with getWizPnP.exe, files 0081 & 0082 were both 32,768kB).
The S1 was running 01.05.350 f/w at the time.

I think it's time for me to give WizZilla a miss until a new version comes out.
I?ll stick with getWizPnP.exe & WizFX, copying in native format so the recordings can be easily checked.

Thanks for all your help Peter. :)
Dennis
U4, Bluey USB tuner, WizTV > Yamaha RX-V3900 > Sony KDL46X2000 TV ||
U4, Bluey USB tuner > Sony KD-43X85J TV > Yamaha YAS-209 || FTA EPG ||
Harmony 650s || (U4s on 19.3.20200901 & T2 on 19.3.20200823) ||
Technicolor DJA0230TLS modem/router, Ethernet LAN, Win10 Home 64 ||

qmanol
Apprentice
Posts: 13
Joined: Sat Jul 24, 2010 21:17

Post by qmanol » Fri Feb 11, 2011 19:34

I'd just like to note that I have been experiencing the same issues with a DP-lite 500GB and the latest firmware available from the site. I did not have any problem with earlier firmware, and the issue only started when I formatted the HDD and reset all settings to defaults, some months after I applied the 01.07.055 update.

What should I use instead?

Additionally, I am interested in a way to schedule an automatic download using command scripts I haven't looked at the getwizpnp docs yet, but is there a way to query each recording in turn and see the name and date of them?

prl
Wizard God
Posts: 32712
Joined: Tue Sep 04, 2007 13:49
Location: Canberra; Black Mountain Tower transmitters

Post by prl » Fri Feb 11, 2011 20:26

qmanol wrote:...I haven't looked at the getwizpnp docs yet, but is there a way to query each recording in turn and see the name and date of them?
Yes, use --list --verbose. But the name and date aren't on the same line.

If you have a Windows box, WizPcl may do more what you want.
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV

qmanol
Apprentice
Posts: 13
Joined: Sat Jul 24, 2010 21:17

Post by qmanol » Fri Feb 11, 2011 21:52

It does seem to, however it refuses to install on my Windows 7 64-bit machine. Based on the output of that command though, I should be able to parse the input in a batch file, or possibly a proper programming/scripting language.

edit: actually it looks like I just need to learn how to use the --expression option., and then I could have it cycle through a text file of episode names and dates.
Last edited by qmanol on Fri Feb 11, 2011 22:18, edited 1 time in total.

prl
Wizard God
Posts: 32712
Joined: Tue Sep 04, 2007 13:49
Location: Canberra; Black Mountain Tower transmitters

Post by prl » Fri Feb 11, 2011 22:07

Sorry, no idea of what you mean by "it" or "that command".
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV

qmanol
Apprentice
Posts: 13
Joined: Sat Jul 24, 2010 21:17

Post by qmanol » Fri Feb 11, 2011 22:20

'it' means WizPcl and 'that command' means getwizpnp --list --verbose.

qmanol
Apprentice
Posts: 13
Joined: Sat Jul 24, 2010 21:17

Post by qmanol » Fri Feb 11, 2011 22:45

I've now developed just enough knowledge of regexp in perl to do what I want, in conjunction with my current storage system. Thanks.

qmanol
Apprentice
Posts: 13
Joined: Sat Jul 24, 2010 21:17

Post by qmanol » Sat Feb 12, 2011 06:37

And now I have some reasons for my wizzilla transfers dropping out constantly - the error I'm getting is "Can't create %filename%: Permission denied" and then on the next line: "Download failed: Forbidden"

I'm now using the latest released version of GetWizPnp. Is there anything I can do to get more useful information about this error?

The command I'm using is getwizpnp.exe -e "/(NCIS|Futurama|Dora The Explorer|Go, Diego|Global Village - Visions of|Good News Week|Handy Manny|House|How I Met Your Mother|Lazy Town|Mythbusters|Rules of Engagement|Talkin' Bout Your Generation|The Goodies).*%%H\s+%%G\s+/i" -d -E -t -R -v -O .\%%G-%%H where %%G and %%H are used to pick a specific date and put all recordings from that date in a specific folder.

prl
Wizard God
Posts: 32712
Joined: Tue Sep 04, 2007 13:49
Location: Canberra; Black Mountain Tower transmitters

Post by prl » Sat Feb 12, 2011 11:05

qmanol wrote:And now I have some reasons for my wizzilla transfers dropping out constantly - the error I'm getting is "Can't create %filename%: Permission denied" and then on the next line: "Download failed: Forbidden"

I'm now using the latest released version of GetWizPnp. Is there anything I can do to get more useful information about this error?

The command I'm using is getwizpnp.exe -e "/(NCIS|Futurama|Dora The Explorer|Go, Diego|Global Village - Visions of|Good News Week|Handy Manny|House|How I Met Your Mother|Lazy Town|Mythbusters|Rules of Engagement|Talkin' Bout Your Generation|The Goodies).*%%H\s+%%G\s+/i" -d -E -t -R -v -O .\%%G-%%H where %%G and %%H are used to pick a specific date and put all recordings from that date in a specific folder.
Hi, qmandol.

I assume that the getWizPnP command you've shown is from a Windows Batch script (the %%X strings look like DOS batch FOR loop variables). I'm not sure how I can help diagnose what's going on if I don't know what might be being substituted into those positions.

The %filename% string you quote above looks like what you might get in a DOS batch script if you use the %filename% variable without having given it a value. Can you extract the actual command being run from you script and post that? That would be very helpful in identifying the problem. Have you tried running a command of the form that the batch script should be running to see whether that works as you intend?

The folder in the -O (--outDir) option must already exist. getWizPnP won't create it if it doesn't exist already. Your batch script will need to create it if it doesn't exist.

It's hard to help when I have to try to guess so much. I'm the author of getWizPnP, so I'm in a fairly good position to help, but I need a bit more to go on. I'm also fairly unfamiliar with DOS batch; that's probably part of the reason why I dislike it so much. Give me bash or even Bourne shell any time :)
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV

qmanol
Apprentice
Posts: 13
Joined: Sat Jul 24, 2010 21:17

Post by qmanol » Sat Feb 12, 2011 12:12

The command is working, it's dropping out halfway through copies, not immediately.

I'll demonstrate by substituting the variables - I am using a for command in a windows batch script to cycle through dates. I don't know if the error is from getwizpnp or from the firmware, but as far as I can tell my syntax is correct, and it is picking the correct programs to copy - it's just failing halfway through some of the copies.

The errors I'm getting are like: "Can't create 3-Feb\Lazy Town.ts: Permission denied", followed by
"Download failed: Forbidden"

The command line is :
getwizpnp.exe -e "/(NCIS|Futurama|Dora The Explorer|Go, Diego|Global Village - Visions of|Good News Week|Handy Manny|House|How I Met Your Mother|Lazy Town|Mythbusters|Rules of Engagement|Talkin' Bout Your Generation|The Goodies).*Feb\s+3\s+/i" -d -E -t -R -v -O .\3-Feb

Which appears to do exactly what I want which is grab any episodes of series matched in the parentheses from the 3rd of February and put them in the 3-Feb folder as TS files.

The folder specified does already exist - The variables that are in the actual script are populated from a dir command which checks the directories available to copy to.

I will rerun the command and post the output as a text file for you.

qmanol
Apprentice
Posts: 13
Joined: Sat Jul 24, 2010 21:17

Post by qmanol » Sat Feb 12, 2011 13:11

Here's a link to the start of the log - it will take a while to run through the whole command but I got lucky and it errored out on the first transfer: http://users.on.net/~kimndrew/BEYONWIZupload.TXT

That file transferred 544MiB before that error came up.

OK, the full log is up - there is an internal server error and a few cannot find device errors in the middle there where I lost network connection for a few seconds. They are not an issue.

prl
Wizard God
Posts: 32712
Joined: Tue Sep 04, 2007 13:49
Location: Canberra; Black Mountain Tower transmitters

Post by prl » Sat Feb 12, 2011 15:50

The error message you're seeing comes from getWizPnP, on the side where it's trying to open a Windows file for appending to put data from the Beyonwiz into. At that point it's not attempting to communicate with the Beyonwiz.

Are the copies all failing after about the same amount of data has been transferred? Do repeated runs of the same copy fail always fail at the same place?

The point where the copy is failing is where getWizPnP opens the output file to append the next data file from the Beyonwiz. In instance the you described, this is after 554MiB has been successfully transferred, which is when it goes to append the 18th data file in the recording.

Could you try a couple more things: one is to try using --resume (or -R) for a copy that's failed (i.e. issue the same command again, but add -R). Does this fail immediately, or does it continue the copy for some time? And if it continues, for how much data? The other is to try a copy without the '-t' option. Does this fail in the same place?

Unfortunately I don't have access to a PC running Windows 7. I'll try running getWizPnP in a similar configuration to what you're trying on an XP machine tomorrow, but in all the recent problems with getWizPnP/WizZilla, I haven't been able to reproduce any of these early termination problems.

I'd suggest upgrading your Lite to 1.07.350 (now available on the Beyonwiz downloads page). Then we'll at least be running the same firmware on the same hardware on the Beyonwiz end. However, I don't think this is a firmware-related problem.
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV

qmanol
Apprentice
Posts: 13
Joined: Sat Jul 24, 2010 21:17

Post by qmanol » Sat Feb 12, 2011 16:34

Thanks for pointing out the firmware, I could have sworn I checked that just a couple of days ago.

Running with the -R option continues the copy. After several times through, the files are all complete. The files don't all stop at the same place. There seems to be no consistency to it.

I do not believe that repeated runs of the same copy fail at the same place, but I will test.
EDIT: Confirmed, the same file that failed last time had no problem this time.


I will check how much has been transferred for each file that failed, then delete the ts files and re-run. I will also then run with resume and note which files continue and which still fail.

Finally I'll try with the tvwiz format. - since it transfers each chunk as a discrete file I suspect it will work perfectly.

I'll log all of these. and get back to you. I shall also try to see if I can find a utility to monitor which processes are accessing the files while the copy is in progress, as it sounds like something might be erroneously causing getwizpnp to think the file is locked. Am I right in understanding that getwizpnp appends each 32MB chunk as a discrete operation?

prl
Wizard God
Posts: 32712
Joined: Tue Sep 04, 2007 13:49
Location: Canberra; Black Mountain Tower transmitters

Post by prl » Sat Feb 12, 2011 17:50

qmanol wrote:... Am I right in understanding that getwizpnp appends each 32MB chunk as a discrete operation?
No, in --ts/-t mode, for each 32MiB recording data file after the first, the destination file is opened read/write, the then getWizPnP seeks to the end, and then writes data blocks (size is controlled by the Perl LWP::Simple module, but much smaller than the whole file) from the HTTP connection to the Beyonwiz to the file. But the error you're seeing is when the file is opened, not for a write to the file.
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV

qmanol
Apprentice
Posts: 13
Joined: Sat Jul 24, 2010 21:17

Post by qmanol » Sat Feb 12, 2011 19:23

Ah, I have expressed myself poorly. What you describe is what I meant - that the file is opened for R/W and then closed for each 32MiB chunk.

I have been running getwizpnp now and everything is working perfectly. The only thing I have changed is that I am now monitoring the filesystem accesses to find what might be opening and locking the files, and I added the --host parameter to the command. So far no failures.

I added the --host parameter because I installed a VM host and the auto-discovery no longer worked with two active network adapters.

My assumption was that some anti-virus or search indexing software was locking the files at just the wrong time, and indeed, each 32MiB after the file is closed, both the indexed and the realtime virus scanner do hit the file, but it's not causing any dropouts now.

prl
Wizard God
Posts: 32712
Joined: Tue Sep 04, 2007 13:49
Location: Canberra; Black Mountain Tower transmitters

Post by prl » Sun Feb 13, 2011 09:50

Hi, qmandol. Nice piece of analysis to go looking for other accesses to the files as they are being written.

I can't see that using the --host option would make any difference to the file creation problem. --host only changes behaviour at the program's startup. Once it has discovered the identity of the Beyonwiz WizPnP server, it uses that IP address for the rest of its execution.

It seems more likely to me that it's the file access monitoring that is somehow causing a change in the way getWizPnP and the virus scanner are interacting - observing the behaviour is somehow changing the behaviour. Werner Heisenberg would be most gratified :)

If you do a copy with both -t and --host, and don't observe the file system accesses, do copies work or no? If they do, you've found a workaround, but I've lost an explanation.

In any case, it seems that the open/append/close method of writing to the .ts file invites nasty interactions with virus scanners, so I think I may change it to be open/append/append/.../close no matter what your testing shows.

Which virus scanner is at work here?

Thanks for your help on this.
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV

sub3R
Wizard
Posts: 2638
Joined: Sun Sep 09, 2007 12:20
Location: Port Macquarie NSW. Reception from Middle Brother.

Post by sub3R » Sun Feb 13, 2011 11:41

qmanol wrote:?.
I shall also try to see if I can find a utility to monitor which processes are accessing the files while the copy is in progress, ?.
qmanol wrote:?.
The only thing I have changed is that I am now monitoring the filesystem accesses to find what might be opening and locking the files, ?.
Hi qmanol. What utility did you use for this?
Dennis
U4, Bluey USB tuner, WizTV > Yamaha RX-V3900 > Sony KDL46X2000 TV ||
U4, Bluey USB tuner > Sony KD-43X85J TV > Yamaha YAS-209 || FTA EPG ||
Harmony 650s || (U4s on 19.3.20200901 & T2 on 19.3.20200823) ||
Technicolor DJA0230TLS modem/router, Ethernet LAN, Win10 Home 64 ||

qmanol
Apprentice
Posts: 13
Joined: Sat Jul 24, 2010 21:17

Post by qmanol » Sun Feb 13, 2011 12:50

Sub3R: I used Process Monitor from Sysinternals which appears to monitor atomic operations by all processes. Beware, it will eat RAM very quickly, as it logs millions of operations over a few minutes. You'll also want to filter the output - each ethernet frame received and written by GetWizPnP is logged as a 1460 byte successful write operation (40 bytes overhead in headers, I assume)

Prl: I use Microsoft Security Essentials. The Windows search indexer was also hitting the file each time it was closed. I did close down Process Monitor halfway through the download process, and still had no dropouts. I don't have a lot of time right now, but I'll try it again without the monitor today if I can.

prl
Wizard God
Posts: 32712
Joined: Tue Sep 04, 2007 13:49
Location: Canberra; Black Mountain Tower transmitters

Post by prl » Sun Feb 13, 2011 12:54

OK, thanks. It's be great if you could have a go at this, too, Dennis.
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV

qmanol
Apprentice
Posts: 13
Joined: Sat Jul 24, 2010 21:17

Post by qmanol » Sun Feb 13, 2011 14:12

Tried both with and without the --host parameter and so far everything is working perfectly now. Sorry I can't provide you with any useful data.

prl
Wizard God
Posts: 32712
Joined: Tue Sep 04, 2007 13:49
Location: Canberra; Black Mountain Tower transmitters

Post by prl » Sun Feb 13, 2011 14:17

IIRC, something similar happened to Dennis, too. But I will re-write the .ts file output to use a single open.
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV

sub3R
Wizard
Posts: 2638
Joined: Sun Sep 09, 2007 12:20
Location: Port Macquarie NSW. Reception from Middle Brother.

Post by sub3R » Sun Feb 13, 2011 20:46

Thanks qmanol for the link & tips on memory & filters.:)
I don?t know why I didn?t think about Process Monitor ? I have an old version on the PC I downloaded some time ago. This is the first time I?ve tried Process Monitor & it crashed while monitoring a download with WizZilla, so I?ll have to download a later version & play around with more of the filters. Before it crashed I noticed I was getting some larger & some smaller bytes written by getWizPnP.exe compared to the regular 1460 bytes written, the larger bytes followed ?Fast IO Disallowed?. This utility does look useful.


Peter, I?m fairly snowed under but when I get some time I?ll see what Process Monitor tells me when WizZilla plays up (getWizPnP.exe wasn?t a problem for me except the known problems with 01.05.334).
I?ve always had search indexing disabled on all of my drives so I know that isn?t an issue.
Dennis
U4, Bluey USB tuner, WizTV > Yamaha RX-V3900 > Sony KDL46X2000 TV ||
U4, Bluey USB tuner > Sony KD-43X85J TV > Yamaha YAS-209 || FTA EPG ||
Harmony 650s || (U4s on 19.3.20200901 & T2 on 19.3.20200823) ||
Technicolor DJA0230TLS modem/router, Ethernet LAN, Win10 Home 64 ||

sub3R
Wizard
Posts: 2638
Joined: Sun Sep 09, 2007 12:20
Location: Port Macquarie NSW. Reception from Middle Brother.

Post by sub3R » Wed Feb 16, 2011 14:39

sub3R wrote:I didn?t try any multiple queued copies using WizZilla because I know they will fail (one thing for certain).
I hadn?t tested this with 01.05.350 f/w even though I got a single recording download to terminate early (without any error message) with the S1 running 01.05.350 f/w, but before I replaced the internal HDD. Running 01.05.320 I typically found the copies stopped downloading somewhere between 2 & 5 queued recordings ? this happened fairly regularly, about every 9 out of 10 times.

Two days ago, running 01.05.350 f/w & a recently installed new internal HDD with an Altronics IDE/SATA adaptor in the S1, I queued 6 recordings & all recordings copied successfully. I then queued 9 recordings, followed by 12 queued recordings, & to my surprise both these groups also copied successfully. This made my early quote sound pretty silly. I then queued 9 recordings, & at the same time or just after the 9th recording started to download, WizZilla displayed the ?Invalid argument to function: at line 6899 Event: TIMER1CTDOWN ? Dialog:0? error message & the 9th recording continued to download successfully in the background. This error message occurred at the same time, or almost the same time, as a timer recording started to record on the S1. Also, for this last group of recordings I applied some bookmarks, renamed one, did a ?keep A-B? edit & partly skipped through the recordings to give some resume points. I don?t know if the timer recording or any of these are relevant but I suspect something that happened prior to or during the download was relevant.

So up to this last failure, with 01.05.350 f/w & the new HDD, I found WizZilla to be a lot more reliable than with 01.05.320 & the old HDD. I don?t think 01.05.350 made a lot of difference because a single recording download terminated early (without any error message) with 01.05.350 prior to the new HDD being fitted, & this only occurred rarely with 01.05.320 f/w & became very difficult to repeat. Did the new HDD make the difference? Had something became corrupted on the old HDD causing the failures? The new HDD was formatted under 01.05.350 f/w & all recordings on the new HDD were new recordings.

Before I removed the old 500GB IDE HDD from the S1 I deleted all recordings & re-indexed the drive. I decided to see if there were any errors on the old drive, but using prl?s PLD Linux Rescue CD for the TF7000HDPVRt I could only find the following:

Code: Select all

[root@rescue /]# fsck ?y /dev/sdd1
fsck 1.39 (29-May-2006)
dosfsck 2.11, 12 Mar 2005, FAT32, LFN
Reclaimed 4 unused clusters (2097152 bytes) in 1 chain.
Free cluster summary wrong (953786 vs. really 953782)
Auto correcting.
Performing changes.
/dev/sdd1: 24 files, 78/953860 clusters.
Yesterday I had some success.

After the above failure, the wiz & the PC were shut down for the night. The next morning I queued the same seven test recordings in WizZilla, started Process Monitor 2.94, & started to download the recordings. The download failed at either the very end of the 2nd recording download or when the 3rd recording was due to start or had just started. The ?Invalid argument to function: at line 6899 Event: TIMER1CTDOWN ? Dialog:0? error message popped up, the word ?calculating? stayed up at the bottom of WizZilla, & the 3rd recording continued to download in the background.

Every attempt (more than 12) to download the above 1st test recording with WizZilla, & also other test recordings resulted in the same type of failure, usually failing just after it had or should have started to download. The download continued in the background each time.
Try as I may, I couldn?t make the download fail with getWizPnP.exe 0.3.4.0 using the same command WizZilla uses. I also couldn?t make the download fail using WizFX (versions 1.0.1.4 & 1.0.2.3). Every time I went back to using WizZilla, the download would fail. This test recording & other recordings I tried failed every time with WizZilla, but I couldn?t make this or any other recording fail with getWizPnP.exe or WizFZ (I never have, except when running 01.05.334 ? which is a known issue).

I have suspected that re-indexing the wiz?s HDD rectifies WizZilla?s download failures for a certain length of time, so I decided to re-index the drive. As soon as the HDD was re-indexed, all downloads with WizZilla have been successful.
Something appears to get screwed up from time to time on the wiz?s HDD that starts these download failures with WizZilla. This is probably why it took some time before I got any failures with WizZilla after the new HDD was installed in the wiz.
Dennis
U4, Bluey USB tuner, WizTV > Yamaha RX-V3900 > Sony KDL46X2000 TV ||
U4, Bluey USB tuner > Sony KD-43X85J TV > Yamaha YAS-209 || FTA EPG ||
Harmony 650s || (U4s on 19.3.20200901 & T2 on 19.3.20200823) ||
Technicolor DJA0230TLS modem/router, Ethernet LAN, Win10 Home 64 ||

sub3R
Wizard
Posts: 2638
Joined: Sun Sep 09, 2007 12:20
Location: Port Macquarie NSW. Reception from Middle Brother.

Post by sub3R » Wed Feb 16, 2011 15:01

Monitoring the downloads with Process Monitor 2.94

With Process Monitor, so far I haven't been able to see when the WizZilla download failure occurs because the current recording being downloaded continues to download in the background after the error message pops up, & I don?t know what the time delay is for TIMER1CTDOWN.
Added to this, Process Monitor collects a lot of data in a relatively short period of time even with some filters being set (it doesn?t take long to get over 100MB in a new log file) - sorting through this & trying to work out what should & shouldn?t happen is very time consuming, especially when trying to compare a successful download with an unsuccessful download, & a download with WizZilla compared to a download with getWizPnP.exe.

WizZilla & getWizPnP do a lot of reading, creating, querying, writing & closing directories & files ? some of these result in ?name not found?, ?path not found?, ?invalid parameter?, ?buffer overflow?, ?no such file? & of course ?success?. Not knowing what should be happening, except for the copying of the files from the wiz to the PC?s drive, I?m just guessing if some of these results are correct or if an error has occurred.
I even noticed one getWizPnP command line change I haven?t seen before (?--dryrun? being added at the end).

Process Monitor showed Norton Antivirus doing its normal work in the background, but it does this during the running of any of the 3 applications (WizZilla, getWizPnP.exe & WizFX).

Something unusual about Process Monitor on my system; after using this in conjunction with WizZilla during a download, the wiz can no longer see any of the shared sub-folders or files on either of the PC?s USB Ext HDDs even though the top level shared folder can be seen. This doesn?t cause any problems on the shared folders on the PC?s two internal HDDs ? these still work normally. The only way to correct this is to reboot the PC, then the wiz can access the shared files in the PC?s Ext HDD?s folders again. Running WizZilla on it?s own without Process Monitor doesn?t create this problem.
Dennis
U4, Bluey USB tuner, WizTV > Yamaha RX-V3900 > Sony KDL46X2000 TV ||
U4, Bluey USB tuner > Sony KD-43X85J TV > Yamaha YAS-209 || FTA EPG ||
Harmony 650s || (U4s on 19.3.20200901 & T2 on 19.3.20200823) ||
Technicolor DJA0230TLS modem/router, Ethernet LAN, Win10 Home 64 ||

prl
Wizard God
Posts: 32712
Joined: Tue Sep 04, 2007 13:49
Location: Canberra; Black Mountain Tower transmitters

Post by prl » Wed Feb 16, 2011 15:11

sub3R wrote:...
I even noticed one getWizPnP command line change I haven?t seen before (?--dryrun? being added at the end).
...
--dryrun is "run, but don't actually transfer/modify/delete any data". I'm not sure what WizZilla uses it for, but for manual use, "--delete --dryrun" can be useful :)
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV

sub3R
Wizard
Posts: 2638
Joined: Sun Sep 09, 2007 12:20
Location: Port Macquarie NSW. Reception from Middle Brother.

Post by sub3R » Wed Feb 16, 2011 20:18

My guess was pretty much on the mark with that Peter. Yes, I can see it would be useful when used in conjunction with delete.
In the command I mentioned, it was used at the end of a recording download command.
Dennis
U4, Bluey USB tuner, WizTV > Yamaha RX-V3900 > Sony KDL46X2000 TV ||
U4, Bluey USB tuner > Sony KD-43X85J TV > Yamaha YAS-209 || FTA EPG ||
Harmony 650s || (U4s on 19.3.20200901 & T2 on 19.3.20200823) ||
Technicolor DJA0230TLS modem/router, Ethernet LAN, Win10 Home 64 ||

BigbobOz
Master
Posts: 305
Joined: Mon May 28, 2007 22:04
Location: Sydney, Inner West

Post by BigbobOz » Sat Feb 19, 2011 14:06

sub3R wrote: I have suspected that re-indexing the wiz?s HDD rectifies WizZilla?s download failures for a certain length of time, so I decided to re-index the drive. As soon as the HDD was re-indexed, all downloads with WizZilla have been successful.
Something appears to get screwed up from time to time on the wiz?s HDD that starts these download failures with WizZilla. This is probably why it took some time before I got any failures with WizZilla after the new HDD was installed in the wiz.
I keep getting "GetWizPnP Process Exited Early" using WizZilla at random points. I can keep resuming and the copy seems fine when playing it back but I have to hit resume constantly

Using WizFX it downloads fine but there's corruptions in the copy.

How do you re-index the HDD? 'Check HDD' in Setup?

I've got a S1 running 01.05.334

User avatar
glow
Guru
Posts: 703
Joined: Fri Feb 15, 2008 11:19
Location: Sydney, Australia DP-P1 DP-S1

Post by glow » Sat Feb 19, 2011 14:25

BigbobOz wrote:I keep getting "GetWizPnP Process Exited Early" using WizZilla at random points. I can keep resuming and the copy seems fine when playing it back but I have to hit resume constantly

Using WizFX it downloads fine but there's corruptions in the copy.

How do you re-index the HDD? 'Check HDD' in Setup?

I've got a S1 running 01.05.334
334 had a major bug which broke WizFX and WizZilla transfers.
You should upgrade to 01.05.350

BigbobOz
Master
Posts: 305
Joined: Mon May 28, 2007 22:04
Location: Sydney, Inner West

Post by BigbobOz » Sat Feb 19, 2011 14:36

Good to know, will do.

Thanks,
Rob

prl
Wizard God
Posts: 32712
Joined: Tue Sep 04, 2007 13:49
Location: Canberra; Black Mountain Tower transmitters

Post by prl » Sat Feb 19, 2011 15:36

BigbobOz wrote:...
How do you re-index the HDD? 'Check HDD' in Setup?
...
FilePlayer>HDD, then SOUNDTRACK. But as glow said, that's probably not the problem.
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV

qmanol
Apprentice
Posts: 13
Joined: Sat Jul 24, 2010 21:17

Post by qmanol » Sat Mar 19, 2011 06:00

prl, just thought I'd mention that I've just had to rebuild my computer from scratch and GetWizpnp is again throwing permission denied errors, so if I can I will find you a reason for that.

prl
Wizard God
Posts: 32712
Joined: Tue Sep 04, 2007 13:49
Location: Canberra; Black Mountain Tower transmitters

Post by prl » Sat Mar 19, 2011 09:50

Thanks. Could you check whether it makes any difference whether you copy in TS file format or Beyonwiz .tvwiz folder format (i.e. with and without the --ts option)?
Peter
T4 HDMI
U4, T4, T3, T2, V2 test/development machines
Sony BDV-9200W HT system
LG OLED55C9PTA 55" OLED TV

qmanol
Apprentice
Posts: 13
Joined: Sat Jul 24, 2010 21:17

Post by qmanol » Sun Mar 20, 2011 20:11

I have success!

The culprit is wmpnetwk.exe aka the Windows Media Player Network Sharing Service. I had disabled it a few hours ago because I noticed it doing a massive amount of reads and writes, presumably building up some sort of index for sharing, which it needed to do because it had just been installed on a computer with a large collection of unindexed media.

Once I fired up Process Monitor everything was working perfectly again, just as it was when I tested last time. Then I recalled that last time I had this issue it was also almost immediately after I had to do a full computer rebuild (new motherboard then, new SSD now), and since I had seen that service hammering my media files it was the perfect candidate. I fired it up while GetWizPnp was still executing and within 5 minutes it had given me a permission denied error, and sure enough, Process Monitor shows that immediately after GetWizPnp closes the file on finishing writing a chunk, wmpnetwk grabs it, and then when GetWizPnp tried to open it to append to the file, it gets a sharing violation, and goes on to the next file.

I have a second of Process Monitor log if you need it but it is nearly 4.5 MiB, and I'm pretty sure I have the problem. It went away automatically for me last time because the indexing finished, and this time because I killed the service. Also, since the issue only occurs when GetWizPnp opens a ts file to append, I can't see that this could ever occur for a .tvwiz format file as it creates a new file for each 32MiB chunk. It should be easily fixable by not closing and opening the file between 32MiB sections.

Post Reply

Return to “Content, Software and USB”