getWizPnP - a command-line WizPnP uploader

[prl]

What hardware/OS are you on? What was the BW doing while you were transferring?

A P4 box running Linux. I worked out why - the perl module that getWizPNP uses honours the http_proxy environment variable so the data was being pushed through a squid proxy which slowed things down significantly. It's now downloading away at around 1Mbyte/s which is slow but better than 100k/s!

That's still slower than it should be.

[prl]

A new version of getWizPnP is available on http://www.beyonwizsoftware.net/.

If you use getWizPnP, you'll need to use this to transfer recordings into valid recording names for Beyonwiz firmware version 1.05.243 (and, presumably later versions) .

Before the next official Beyonwiz firmware release, you'll also need to check any recordings you've transferred using getWizPnP. Any characters in the recording names (both .ts and Beyonwiz .tvwiz folder format) that aren't legal in Windows filenames ( \ / : * ? " < > | ) will need to be removed or replaced. Any recordings transferred in Beyonwiz .tvwiz folder format (without using the --ts flag) will need to have the extension .tvwiz added to their name.

My apologies, but the rules used by the Beyonwiz to recognise recordings on Windows Shares has changed in 1.05.243.

Bug fixed in 0.2.1:

• The wrong parts of recording chunks were being copied in some cases of edited and instant recordings (transferred without --ts flag) -- fixed.
• Now eliminates file name characters that are illegal in Windows from all recording names, even on systems where those characters are legal. This is for compatibility with the File Player for Beyonwiz firmware 1.05.243 (and later, presumably)
• Now eliminates adds .tvwiz extension to all Beyonwiz format recordings (recordings transferred without --ts flag). This is for compatibility with the File Player for Beyonwiz firmware 1.05.243 (and later, presumably)

[prl]

,,,
PS. How's the delete switch coming along?..

...
Delete wouldn't be hard to implement, but I have some concerns about side-effects on the BW. Deleting a recording that's currently being viewed is likely to cause the BW a bit of indigestion As far as I know there's no flag available through WizPnP to tell if a recording is being played.

I will try it to see how nasty it gets.

I deleted a recording while it was being played (via a telnet login, not with getWizPnP), and it had no ill effects (apart from the enjoyment of the viewer ). The BW flashed up an error message (too quickly to read), and went straight on to playing the next recording.

The stat file has no indication in it that the file is being played (a stat file was the same before, during and after playing the recording).

[jrx]

Thanks Peter, so the delete functionality is not too far away?

Is there any reason the H1 can't delete recordings on a P1 WixPnP server?

[prl]

Thanks Peter, so the delete functionality is not too far away?

Delete will probably be added to getWixPnP, but I want to do a bit more investigation, especially some more tests of deleting recordings while they're playing, and trying to see if there's any marker in the header files to indicate that a recording is being played.
I'm not setting a timeframe for implementation.

Is there any reason the H1 can't delete recordings on a P1 WixPnP server?

No idea. There used to be a File Player POPUP menu item to delete recordings on a WizPnP server, but it didn't work. The recording would disappear from the client's list of recordings but would remain on the server. That bug has been "fixed" in recent firmware (at least on the H1) by disabling the Delete entry in the menu.

[efry]

Delete will probably be added to getWixPnP, but I want to do a bit more investigation, especially some more tests of deleting recordings while they're playing, and trying to see if there's any marker in the header files to indicate that a recording is being played.
I'm not setting a timeframe for implementation.

Hi Peter,

I was wondering how do you were planning to delete files? Is that part of the uPnP spec? I couldn't find any mention of deletion in the wizfx protocol description.

I'd be interested in adding delete capability to WizMac too.

Regards,
Eric

[prl]

Delete will probably be added to getWixPnP, but I want to do a bit more investigation, especially some more tests of deleting recordings while they're playing, and trying to see if there's any marker in the header files to indicate that a recording is being played.
I'm not setting a timeframe for implementation.

Hi Peter,

I was wondering how do you were planning to delete files? Is that part of the uPnP spec? I couldn't find any mention of deletion in the wizfx protocol description.

I'd be interested in adding delete capability to WizMac too.

Regards,
Eric

Spec? Schmec!

/flash/wizdvp/wizpnp, which does the WizPnP server hard lifting, has shttpd built in. Shttpd supports the HTTP DELETE operation. Its implementation of HTTP DELETE uses the library function remove(3), which allows empty directories to be removed as well as files. Problem solved*

I already use HTTP DELETE and HTTP PUT in bwhack.pl to create and delete flag files on the HDD to control whether hacks are enabled or disabled. HTTP PUT is broken, though, and only allows you to create empty files, otherwise I could have some real fun with that capability

Unfortunately, a lot of the other shttpd features are optioned out of the BW build. CGI would have been great - run remote scripts; pass them simple parameters and all!.

* Well, almost solved:

• Because it's outside the spec, there's no way to tell whether a recording is being played. I've done one test of blowing away a recording while it was being played, and the BW coped admirably, but I don't know whether that's guaranteed. I'll have to try deleting a recording on the WizPnP server while it's being streamed, see what that does, too.
• Although it looks as though the BW shttpd has directory listing compiled in, it is disabled, and I haven't been able to enable it using a /etc/shttpd.conf file, so if the recording has unexpected files in it (as happens with the "Error loading media file" bug), then the procedure of remove header.tvwiz, stat, trunc and all the files listed in trunc won't work properly, because it won't empty the directory, and the directory won't be deleted.

[magicRob]

How's this going? Any idea on a delete function?

Basically while I'm away on holidays, what I want to setup is a script to go and move the previous days recordings off the BW onto some other storage to keep space free...

I'm capable of coding, care to share what you have already to do the deleting?

Update Found this document:

http://svn.openwiz.org/svnroot/wizfx/tr ... WizPnP.pdf

Hopefully my PHP skills can let me put something together in the next week or so I want to have space to keep recording Am getting a WRT54GL to put behind my TV cabinet tonight to give it wifi capability as well as networking capabilities with other things sitting in my cabinet

[IanSav]

Hi Peter,

While this thread is being brought to the top...

Has Beyonwiz implemented the shttpd source code fixes you suggested?

Regards,
Ian.

[prl]

How's this going? Any idea on a delete function?

Basically while I'm away on holidays, what I want to setup is a script to go and move the previous days recordings off the BW onto some other storage to keep space free...

I'm capable of coding, care to share what you have already to do the deleting?

Update Found this document:

http://svn.openwiz.org/svnroot/wizfx/tr ... WizPnP.pdf

Hopefully my PHP skills can let me put something together in the next week or so I want to have space to keep recording Am getting a WRT54GL to put behind my TV cabinet tonight to give it wifi capability as well as networking capabilities with other things sitting in my cabinet

The main thing that was holding me back was not being sure what would happen if a recording was deleted while it was being played. I've since discovered that nothing bad happens apart from the playback coming to an abrupt end. I wasn't intending allowing current recordings to be deleted.

One other problem is the old (now disguised, but probably still present) bug where one of the header files is created with the wrong name. The incorrectly-named file is usually a fixed and known name, but it's occasionally popped up under a different name. If all the files that make up a recording can't be deleted, then the folder than contains the recording can't be deleted, and so is likely to show up as something weird in the BW GUI. This will probably be an acceptably rare problem. Unfortunately, the HTTP server in the BW doesn't allow listing the contents of a directory to find out what's in it.

I also want to try to get it to disable its WizPnP server search when it's run in "bare" Windows (rather that under Cygwin), because the ActivePerl package library doesn't have the components needed to do the SSDP search to find the WizPnP servers.

The main problem I have is time to do the work to implement it. I got some work done on my BWFWTools last weekend, but I'm not sure when i can get a new release of either out the door.

If you want to try to code it yourself, the latest release of getWizPnP is available for download on beyonwizsoftware.net. To delete recordings, all you need to do is use the HTTP DELETE command on all the files listed in the trunc file (code to load and interpret the trunc file is already part of getWizPnP), then delete the header files: header.tvwiz, trunc, stat and at and then the folder containing the recording. When the stat file name gets mangled, it usually is to "at".

But the code isn't PHP, it's Perl.

[prl]

Hi Peter,

While this thread is being brought to the top...

Has Beyonwiz implemented the shttpd source code fixes you suggested?

Regards,
Ian.

Haven't got around to testing yet. I don't even have telnet enabled on my S1 at the moment. I've been meaning to try for a while. It would be nice to be able to put files to the BW; especially so with my httproot hack, which I think would let me put files anywhere, including updating /tmp/config/rc.local remotely

[magicRob]

I realise it's Perl, I was going to try and write something in PHP as I'm far more proficient in it

[prl]

Hi Peter,

While this thread is being brought to the top...

Has Beyonwiz implemented the shttpd source code fixes you suggested?

Regards,
Ian.

Not as far as I can tell.

[IanSav]

Hi Peter,

Has Beyonwiz implemented the shttpd source code fixes you suggested?

Not as far as I can tell.

Regards,
Ian.

[prl]

There's a new version of WizPnP in alpha test at the moment (thanks, alwayslooking and Dave?).

The main new features are:

• Delete recordings on the Beyonwiz or on the local disk
• Move recordings (copy/delete) from the Beyonwiz (or locally). Local moves are done more efficiently than copy/delete where possible.
• Optionally add episode name to recording folder name (if the episode name is set).
• Added current transfer rate (and average rate at completion) to the progress bar (--verbose).
• More graceful handling of the module IO::Socket::Multicast not being available on the system (Windows ActivePerl PPM library doesn't contain IO::Socket::Multicast).
• Fixed a bug where, if multiple selection arguments were on the command line, only the first was used.

I need to do some Windows testing, and then it should be ready for release, probably next weekend 18/19 Oct.

I have a question for Windows users.

At the moment in the new version, if you try to use the WizPnP device search mechanism and IO::Socket::Multicast is missing, getWizPnP exits with a message that tells you to either get IO::Socket::Multicast or use --host to specify your Beyonwiz device. Unfortunately, the most popular Windows Perl implementation, ActivePerl, doesn't currently have IO::Socket::Multicast in its PPM package management system library.

It would be possible to make WizPnP device search work (after a fashion) even without IO::Socket::Multicast, because WizPnP servers actively advertise themselves once a minute, even when there hasn't been a multicast search request, so within a minute, getWizPnP could discover all the WizPnP servers on the local net.

Is it worth implementing this? Personally, I probably wouldn't use it (I'd set $host in my $HOME/.getwizpnp file), I think it's a bit too slow, especially for --list requests. What does the forum think?

[alwayslooking]

I have a question for Windows users.

At the moment in the new version, if you try to use the WizPnP device search mechanism and IO::Socket::Multicast is missing, getWizPnP exits with a message that tells you to either get IO::Socket::Multicast or use --host to specify your Beyonwiz device. Unfortunately, the most popular Windows Perl implementation, ActivePerl, doesn't currently have IO::Socket::Multicast in its PPM package management system library.

It would be possible to make WizPnP device search work (after a fashion) even without IO::Socket::Multicast, because WizPnP servers actively advertise themselves once a minute, even when there hasn't been a multicast search request, so within a minute, getWizPnP could discover all the WizPnP servers on the local net.

Is it worth implementing this? Personally, I probably wouldn't use it (I'd set $host in my $HOME/.getwizpnp file), I think it's a bit too slow, especially for --list requests. What does the forum think?

The only reason I can think to do this is if someone uses getwizpnp in a script or app that needs to find the beyonwiz rather than specify it...and you may only need to do this if the IP Address on the wiz changed constantly (DHCP).

[IanSav]

Hi Peter,

Is it worth implementing this? Personally, I probably wouldn't use it (I'd set $host in my $HOME/.getwizpnp file), I think it's a bit too slow, especially for --list requests. What does the forum think?

Is this something that can be done as a backgrounded thread or task? The results can be used when they become available, unless the user fills in a specific address first.

Regards,
Ian.

[prl]

Hi Peter,

Is it worth implementing this? Personally, I probably wouldn't use it (I'd set $host in my $HOME/.getwizpnp file), I think it's a bit too slow, especially for --list requests. What does the forum think?

Is this something that can be done as a backgrounded thread or task? The results can be used when they become available, unless the user fills in a specific address first.

Regards,
Ian.

It could be, but there's not much point; it's a command line interface, and it can't do anything until it has the details for the device you want to use. For a GUI it probably would be a reasonable way to go.

[IanSav]

Hi Peter,

It could be, but there's not much point; it's a command line interface, and it can't do anything until it has the details for the device you want to use. For a GUI it probably would be a reasonable way to go.

Perhaps while you are at the prompt waiting for the user to key in the target Beyonwiz device the enumeration could be happening. When the results come in abort the input and revert to a list selection menu. (Hey, its an idea. ) This way the user can type in the address immediately or get a selection list if they wait for a few seconds (or so).

Edit: Another idea is to ask if they want to enter a Beyonwiz address or scan for Beyonwiz units. That way there is immediate response from the program and an opportunity to enter the name of a specific device or take a minute to just scan for all available devices.

Regards,
Ian.

[prl]

Hi Peter,

It could be, but there's not much point; it's a command line interface, and it can't do anything until it has the details for the device you want to use. For a GUI it probably would be a reasonable way to go.

Perhaps while you are at the prompt waiting for the user to key in the target Beyonwiz device the enumeration could be happening. When the results come in abort the input and revert to a list selection menu. (Hey, its an idea. ) This way the user can type in the address immediately or get a selection list if they wait for a few seconds (or so).

Edit: Another idea is to ask if they want to enter a Beyonwiz address or scan for Beyonwiz units. That way there is immediate response from the program and an opportunity to enter the name of a specific device or take a minute to just scan for all available devices.

Regards,
Ian.

The user doesn't type anything into getWizPnP. It's a monolithic command line tool, that takes a list of selectors for whatever operation it's doing (list, copy, move, delete), it gets the Beyonwiz address (either fixed IP address from the commandline or from .getwizpnp, or via a WizPnP search), then contacts the Beyonwiz for index.txt, and then starts loading the necessary parts of the header.tvwiz files to know which recordings have been selected. It does this each time it's run.

Users can already select a Beyonwiz IP address as an option on the command line or in the .getwizpnp file. It's also possible to select a Beyonwiz WizPnP server name on the command line, but that still needs a WizPnP search to be done.

Have a look at the manual (the download has a HTML of the manual in getWizPnP/html/getWizPnp.html).

[alwayslooking]

It could be, but there's not much point; it's a command line interface, and it can't do anything until it has the details for the device you want to use. For a GUI it probably would be a reasonable way to go.

It might be useful for the Windows based GUI we're doing, but at this stage we require the user to enter the IP Address.

Can we look at it once we've finished playing with your new version and working around the changes to WizPnP, which I think we may have sorted out?

[prl]

New version (0.3) of getWizPnP posted on beyonwizsoftware.net.

New:

• Delete recordings on the Beyonwiz or on the local disk
• Move recordings (copy/delete) from the Beyonwiz (or locally). Local moves are done more efficiently than copy/delete where possible.
• When the --date option is used, the date is separated from the title by ' - ', rather than simply by a single space
• Optionally (--episode) add episode name to recording folder name (if the episode name is set in the recording header). The episode name is also separated from the recording name by ' - ' if there is an episode name in the recording header.
• The recording name is constructed as title[ - date][ - episode]
• Added current transfer rate (and average rate at completion) to the progress bar (--verbose).
• More graceful handling of the module IO::Socket::Multicast not being available on the system (Windows ActivePerl PPM library doesn't contain IO::Socket::Multicast).
• Fixed a bug where, if multiple selection arguments were on the command line, only the first was used.
• Handles recordings in subfolders (available in firmware versions 01.05.269 beta and later)

Bugs:

• Uses bignum package for 64-bit integers, even when the underlying Perl integers are 64 bits.
• When resuming a download, may fetch up to 32MB more data than is necessary.
• WizPnP device search is not supported when used in a Windows Command Prompt window. WizPnP search works correctly on Linux, MacOS X and Cygwin on Windows.
• It is unknown whether deleting a recording that is currently being watched can cause any problems on the Beyonwiz. Normally, if a recording is deleted while it is being watched, the playback stops with an error message and returns the Beyonwiz user to the File List
• Subfolder handling is primitive. All recordings in all subfolders are treated in the same way. Subfolder selection is not possible. This may be fixed before a Beyonwiz firmware release that allows subfolder access is made public.
• Does not allow operations on media files from the Beyonwiz contents folder (access to the contents folder is available only in Beyonwiz firmware versions 01.05.271 and later).
• If a recording is copied or moved on the Beyonwiz, the moved recording "disappears" from getWizPnP, and the new copy cannot be accessed. This is a Beyonwiz firmware bug. Workaround is to make a new recording on the Beyonwiz, delete a recording on the Beyonwiz (not using getWizPnP[/i]), or to restart the Beyonwiz.

Fixed:

• In earlier versions, if multiple selection arguments were on the command line, only the first was used. Fixed.

[tonymy01]

Nice one Peter!

[alwayslooking]

That's great Peter.
I also noticed you added a "-" to the append date option in this version.

[prl]

That's great Peter.
I also noticed you added a "-" to the append date option in this version.

Yes, and one gets put in front of the episode name, too if it is added.

I forgot to add it to the list of features. Will add it soon.

Edit: done.

[prl]

A couple of new options for getWizPnP, for the next release:

--folder=foldername
-f folder

Restrict the operations to the named folder. More that one --folder option may be given, the operations apply to all the named folders. If no folders are names, operations are on all recordings in all folders.

--sort=sortcode
-s sortcode

Sort the output according to the value of sortcode.
Sortcode is a string made up of the codes:
uu (unsorted),
az (title alphabetic ascending),
za (title alphabetic descending),
fa (folder name alphabetic ascending),
fd (folder name alphabetic descending),
ta (date and time ascending),
td (date and time descending),
da (date ascending) and
dd (date descending).
The default sortcode is faazta. Later codes in the string are only checked if entries are equal on all earlier codes. The default sortcode sorts on folder name, then title (not including episode title),
then from earliest to latest for the same title & folder name.

The sorting on the title uses a dictionary/book index style of sort order. Case, punctuation and spaces are ignored, if the title starts with "MOVIE: ", that is ignored, and if the (remaining) title starts with "a", "an" or "the", that is ignored.

Case is not ignored for the folder name sort order.

If the uu code is used anywhere in the string, the output is unsorted.

--------------------
I'm looking for some opinions:

• Should --folder select only the named folder, or the named folder and subfolders?
• Should folder name selection ignore case? It does at the moment.
• If subfolders can be included, should they be included by default and excluded by option, or excluded by default and included by option?
• faazta and azta appear to be the most useful defaults for sorting. Which one? The default can be changed in the user's .getwizpnp file anyway.
• Does the sorting on title sound right? Should it ignore "MOVIE:" and articles at the beginning?
• Should folder name sorting ignore case? It doesn't at the moment. This should probably be the same as --sort.

[IanSav]

Hi Peter,

Should --folder select only the named folder, or the named folder and subfolders?

If you don't mention any folders then process all folder recursively. If a folder is listed then only process *that* folder. (See note about recursion later.)

Should folder name selection ignore case? It does at the moment.

Given that this is a tool that works with the Beyonwiz using a Microsoft view of the world I would suggest that folder case should be preserved (as defined by the folder on the disk not the folder specified on the command line) but the selection should be case insensitive.

If subfolders can be included, should they be included by default and excluded by option, or excluded by default and included by option?

As mentioned above, if folders are mentioned then subfolders should be ignored. This behaviour could be modified by adding a new option for recursion. If a folder is mentioned *and* the recursion option is specified then that folder and all sub folders should be processed.

faazta and azta appear to be the most useful defaults for sorting. Which one? The default can be changed in the user's .getwizpnp file anyway.

If the default is settable by the user this question is irrelevant.

Does the sorting on title sound right? Should it ignore "MOVIE:" and articles at the beginning?

This sort of manipulation always comes back to bite you in the bum. I can sort of see a point for the "MOVIE: " as this seems to be a common filler for Channel 9. The other words would not be my choices for stripping out. If this is to be done then I would prefer that it only happen if the user specifically asks for it to happen. I would also suggest the ignore word list also be in the user's initialisation file.

Should folder name sorting ignore case? It doesn't at the moment. This should probably be the same as --sort.

The case for creating and sorting folders and files should be the same as that found on the source. Using those files and folders should be case insensitive as that is the way the Beyonwiz already works.

Regards,
Ian.

[prl]

...
Given that this is a tool that works with the Beyonwiz using a Microsoft view of the world...

Harrumph! Almost all of the development of the tool is done on Mac OS X. getWizPnP works least well on Windows (because I can't find a copy of IO::Socket::Multicast that works properly).

Other comments noted. I'll wait for more feedback before I reply.

[prl]

Hi Peter,

Should --folder select only the named folder, or the named folder and subfolders?

If you don't mention any folders then process all folder recursively. If a folder is listed then only process *that* folder. (See note about recursion later.)

At the moment --folder only selects that folder and not subfolders. I'll add an option to include subfolders.

Should folder name selection ignore case? It does at the moment.

Given that this is a tool that works with the Beyonwiz using a Microsoft view of the world I would suggest that folder case should be preserved (as defined by the folder on the disk not the folder specified on the command line) but the selection should be case insensitive.

Folder selection is currently case-insensitive. Folder names aren't used on the client side. If they are used in the future they will preserve case to the extent that the host system allows it.

If subfolders can be included, should they be included by default and excluded by option, or excluded by default and included by option?

As mentioned above, if folders are mentioned then subfolders should be ignored. This behaviour could be modified by adding a new option for recursion. If a folder is mentioned *and* the recursion option is specified then that folder and all sub folders should be processed.

Yes.

faazta and azta appear to be the most useful defaults for sorting. Which one? The default can be changed in the user's .getwizpnp file anyway.

If the default is settable by the user this question is irrelevant.

I don't think it's entirely irrelevant; I think it should work out of the box with minimum reconfiguration. However, no-one has expressed any opinion on it, so I guess no-one cares too much.

Does the sorting on title sound right? Should it ignore "MOVIE:" and articles at the beginning?

This sort of manipulation always comes back to bite you in the bum. I can sort of see a point for the "MOVIE: " as this seems to be a common filler for Channel 9. The other words would not be my choices for stripping out. If this is to be done then I would prefer that it only happen if the user specifically asks for it to happen. I would also suggest the ignore word list also be in the user's initialisation file.

"MOVIE: " may only be used as a filler on Nine on FTA, but it appears to be used for movies on all channels in IceTV's EPG.

I'm puzzled by your saying that the/a/an wouldn't be your choices for ignoring for sort order. I'm only suggesting that they are ignored at the start of a title, not as a general stop list. This is pretty common practice for indexes.

Anyway, the sorting on the Beyonwiz is case-insensitive, but otherwise it compares all characters, including spaces and punctuation, and of course doesn't ignore the "MOVIE: " prefix. So that's what the default will be. There will be command line options (and corresponding config file variables) to allow more dictionary-like sorting. The list of words ignored at the start of a title will be configurable, but default (if this feature is used) to the/a/an.

Should folder name sorting ignore case? It doesn't at the moment. This should probably be the same as --sort.

The case for creating and sorting folders and files should be the same as that found on the source. Using those files and folders should be case insensitive as that is the way the Beyonwiz already works.

Regards,
Ian.

I think that's what I've already implemented, but if not it's how the release will be implemented.

Thanks for the feedback.

[IanSav]

Hi Peter,

I think I miss-understood/miss-read your meaning of the the/a/an filter. I originally read it to mean that these words would be stripped out. I now understand that you just want to ignore the words for sorting purposes only. Sorry 'bout that chief.

Regards,
Ian.

[prl]

Hi Peter,

I think I miss-understood/miss-read your meaning of the the/a/an filter. I originally read it to mean that these words would be stripped out. I now understand that you just want to ignore the words for sorting purposes only. Sorry 'bout that chief.

Regards,
Ian.

No problem. I wondered whether that was the case. A general stop list, I agree, would be a good bit bigger.

[prl]

New versision of getWizPnP released (0.3.1).

This is in preparation for the new version of the firmware expected shortly. Some of the new features depend on capabilities that are not in 1.05.261. However, this version should be able to do everything that you could do with 0.3 on Beyonwiz firmware 1.05.261. 0.3.1 is a good bit faster than 0.3 in listing the recordings on the Beyonwiz and may be worth using just for that reason.

Release notes

New:

• New options --sort, --dictionarySort and --dictStoplist for controlling sorting.
• New options --folder and --recursive to select the folder(s) being accessed
• New option --Verbose=n to explicitly set the verbosity level of the output
• The "Index name" entry is now only printed for --Verbose=3 or greater
• The "Index name" entry can be added to the output for --Verbose=0 to --Verbose=2
  by using the new --index option (less cluttered output for normal verbosity levels).
• The order in which extra information is provided with increasing verbosity has
  changed for --Verbose=3 and above, and this is now documented.
• Listing the recordings on the Beyonwiz is now faster, because of improvements in the "lazy loading" of data from the recording header files
• The transfer rate for copy or move shown with --Verbose=1 or greater is now averaged over the last 20MB transferred.

Bugs:

• Uses bignum package for 64-bit integers, even when the underlying Perl integers are 64 bits.
• When resuming a download, may fetch up to 32MB more data than is necessary.
• WizPnP device search is not supported when used in a Windows Command Prompt window. WizPnP search works correctly on Linux, MacOS X and Cygwin on Windows.
• Minor know problems that deleting a recording can cause are documented in the BUGS section of the docuentation. It is not known whether deleting a recording using getWizPnP can cause any major problems on the Beyonwiz.
• Does not allow operations on media files from the Beyonwiz contents folder (access to the contents folder is available only in Beyonwiz firmware versions 01.05.261 and later).
• If a recording is copied or moved on the Beyonwiz, the moved recording "disappears" from getWizPnP, and the new copy cannot be accessed. This is a Beyonwiz firmware bug. Workaround is to make a new recording on the Beyonwiz, delete a recording on the Beyonwiz (not using getWizPnP[/i]), or to restart the Beyonwiz. This bug is expected to be fixed in the next public release of the Beyonwiz firmware (the one following 1.05.261).
• Folder operations only work on firmware more recent than 1.05.261
• Sort on title only works on firmware more recent than 1.05.261

Fixed:

• Some formatting problems in the documantation fixed.

[prl]

getWizPnP source code is now also available on OpenWiz.

[craigh]

Hi Peter

Had not played much with getWizPnP till I took the compiled version out of WizZilla

It is an excellent utility.

Have set up a number of automatic moves from my BW to my NAS for certain evenings.

One question I have relates to the large number of TVWiz directories I currently have on the NAS. Given the BW now seems to play TS files without a problem (main reson I chose TVWiz format before) I wanted to convert existing recordings to TS format.

However I like the way your tool extracts the episode name and appends date etc.

Is there a way to use getWizPnP to work against shows on file system to do this conversion ?

[prl]

Hi Peter

Had not played much with getWizPnP till I took the compiled version out of WizZilla

It is an excellent utility.

Thanks

... I wanted to convert existing recordings to TS format.

However I like the way your tool extracts the episode name and appends date etc.

Is there a way to use getWizPnP to work against shows on file system to do this conversion ?

Mais naturellement

Use the --indir option to set the local source directory. --recursive doesn't work (yet) on local directories, though.

Rod Whitby (rwhitby) has emailed that he's had some problems (no audio on BW playback) with BW recordings converted to .TS files using getWizPnP, so it may be a good idea to check that your conversion worked before you delete the source. --move --ts should work if you decide to wing it

[prl]

Hi Peter

Had not played much with getWizPnP till I took the compiled version out of WizZilla
...

I've been thinking of putting a compiled version for Windows in the beonwizsoftware.net distribution. I just need to find out how to do it.

[craigh]

Hi Peter

I gave indir a go and had limited success.

The following returns a list of shows

D:\Downloads\Beyonwiz\getWizPnP>getwizpnp.pl --indir=\\hhomens\beyonwiz\bones -V
3 -f / --outdir=d:\ -L
Bones Dec.29.2008_18.26
Bones_1 Dec.29.2008_18.26
Bones_10 Dec.29.2008_18.26
Bones_11 Dec.29.2008_18.26
Bones_13 Dec.29.2008_18.26
Bones_14 Dec.29.2008_18.26
Bones_15 Dec.29.2008_18.26
Bones_16 Dec.29.2008_18.26
Bones_17 Dec.29.2008_18.26
Bones_18 Dec.29.2008_18.26
Bones_2 Dec.29.2008_18.26
Bones_20 Dec.29.2008_18.26
Bones_21 Dec.29.2008_18.26
Bones_22 Dec.29.2008_18.26
Bones_24 Dec.29.2008_18.26
Bones_25 Dec.29.2008_18.26
Bones_27 Dec.29.2008_18.26

However I cant seem to get it to start the copy process, it just takes the commend and returns to the prompt a second later


D:\Downloads\Beyonwiz\getWizPnP>getwizpnp.pl --indir=\\hhomens\beyonwiz\bones -V
3 --folder=/ --outdir=d:\ --ts Bones

Tried lots of variants.

Same sort of command or leave out the show name and it works fine against a BW.

[prl]

I hope I haven't broken something. I'll give it a go.

Edit: I tried it on my Mac, perl version, using as close as possible arguments, and it works OK for me:

Code: Select all

getWizPnP.pl --indir=/Volumes/Archive/Archive/Tests --outdir=/Volumes/Archive/Archive/Tests -V 3 --folder=/ --ts Coup
WIN TV HD: Sea Patrol II: The Coup_save2 - Copy
    Index name: Sea Patrol II_ The Coup_save2_Apr.14.2008_20.46 Dec.29.2008_19.49
    Mon Apr 14 20:46:05 2008 - Mon Apr 14 20:48:04 2008
    playtime:    1:59    recording size:    158.3 MB
|==================================================|  6.2MB/s 100% 158/158MB

WIN TV HD: Sea Patrol II: The Coup_save3 - Copy
    Index name: Sea Patrol II_ The Coup_save3_Apr.14.2008_21.39 Dec.29.2008_19.49
    Mon Apr 14 21:39:54 2008 - Mon Apr 14 21:42:00 2008
    playtime:    2:06    recording size:    178.3 MB
|==================================================|  5.4MB/s 100% 178/178MB

I'll have to move some recordings over to a PC and give it another go.

[prl]

...

Code: Select all

D:\Downloads\Beyonwiz\getWizPnP>getwizpnp.pl --indir=\\hhomens\beyonwiz\bones -V 3 --folder=/ --outdir=d:\  --ts Bones

Tried lots of variants.
...

I've been wondering whether I have handled the UNC name correctly. Could you try with a drive letter instead of \\hhomens?

[craigh]

...

Code: Select all

D:\Downloads\Beyonwiz\getWizPnP>getwizpnp.pl --indir=\\hhomens\beyonwiz\bones -V 3 --folder=/ --outdir=d:\  --ts Bones

Tried lots of variants.
...

I've been wondering whether I have handled the UNC name correctly. Could you try with a drive letter instead of \\hhomens?

Same outcome after i mapped a drive

I have tried directories on the local machine as well with same outcome....it runs for a few seconds then returns to prompt.

C:\Documents and Settings\Hart Home\My Documents\Craig's Files\Home Theater\getW
izPnP>getwizpnp --indir=x:\bones --folder=/ --outdir=d:\temp --ts -v

C:\Documents and Settings\Hart Home\My Documents\Craig's Files\Home Theater\getW
izPnP>

[prl]

Craigh, I've been able to reproduce your problem, found the bug and fixed it. The problem was in the code that extracted the path name from the index.txt data (I construct index.txt data for local directories - it was the simplest way to get local recordings working). My code expected a '/' path separator, which works on Linux, MacOS X and Cygwin/Win32, but not on naked Win32.

I've fixed the problem and the fix will be in the next release. If you want, I can check the fix into the subversion repository so you could get a fix by doing an svn export from the repository, but to make it work, you need to install Perl on yout machine.

Otherwise you'll have to wait until there's a new getWizPnP distribution and alwayslooking and Dave? distribute a Windows EXE of it with WizZilla.

There are a number of other things I want to do before I push out a new distribution: make --recursive work for local files and add caching to WizPnP device search. Possibly also do a Windows EXE version of getWizPnP, but that may not make it to the next distribution. But perhaps I'll put caching WizPnP search on hold, and push out the local file updates. --recursive should be fairly easy.

[craigh]

Thanks Peter

Thanks for the update.

I did install Perl to check if it was an issue with the compled version so I could use the normal version when its ready.

Just as an aside i had it move 2 files last nigh (Bones) and was surprised today to find they (TS) were both corrupt in some way.

If I tried to load them on teh BW it tool minutes then loaded with no navigation.

Tried to open them in VideoRedo and it failed.

Very strange as all my testing yesterday worked fine (however I was using local drives rather than SMB)....I will test more today.

Shame I used MOVE as now I cant check teh original for errors

This is the output from Project X -t truncated the file to 11 minutes.

<<< end of quick info >>>

-> Input File 0: \\hhomens\beyonwiz\Bones\Bones - The Man in the Fallout Shelter - Mon Dec 29 2008.ts (5905186816 bytes)
-> File is DVB/MPEG-2 TS (Video/Audio/TTX PES)
-> make a TS
-> Service ID 0x960
-> PMT 0x1CC refers to these usable streams:
Video: PID: 0xAB4(#0)
Audio: PID: 0xAB5(eng)(#138)
Teletext:
Subpict.:

-> special PIDs for searching defined
ok> PID 0xAB4 has PES-ID 0xEA (MPEG Video) mapped to 0xE0 (79900 #426)
ok> PID 0xAB5 has PES-ID 0xC0 (MPEG Audio) mapped to 0xC0 (150212 #800)
!> PID 0xAB4 -> packet 887550 @ pos. 166859212 out of sequence (13/12) (shifting..) (~00:00:00.000)
!> PID 0xAB5 -> packet 887553 @ pos. 166859776 out of sequence (4/11) (shifting..) (~00:00:00.000)
!> PID 0xAB4 -> packet 1780060 @ pos. 334651092 out of sequence (13/11) (shifting..) (~00:00:00.000)
!> missing sync. byte @ 403701612
!> found sync. byte @ 436207752
!> PID 0xAB4 -> packet 2147350 @ pos. 436207752 out of sequence (8/9) (shifting..) (~00:00:00.000)
!> PID 0xAB5 -> packet 2147369 @ pos. 436211324 out of sequence (9/2) (shifting..) (~00:00:00.000)
!> incomplete ts packet at EOF detected @ 5905186784
packs: 3348934 100% 5905186816
---> new File: E:\Video Processing\Bones - The Man in the Fallout Shelter - Mon Dec 29 2008(ts).ts

summary of created media files:
Stream: 'E:\Video Processing\Bones - The Man in the Fallout Shelter - Mon Dec 29 2008(ts).ts'
=> 642033536 bytes written...

[prl]

Pity about using --move

If you find any more recordings where this happens, let me know and I'll see what I can dig out about it.

[craigh]

Hi Peter

Not sure if this may be a clue to the problem but both recordings were Prime HD and both changed from 720*576 to 1440*1080 after a few minutes of recording.

THats not a big issue with the BW but I know VideoRedo does not seem to like joining the TVWiz files when that is the case.

Possible the TS creation is the same ?

[prl]

Hi Peter

Not sure if this may be a clue to the problem but both recordings were Prime HD and both changed from 720*576 to 1440*1080 after a few minutes of recording.

THats not a big issue with the BW but I know VideoRedo does not seem to like joining the TVWiz files when that is the case.

Possible the TS creation is the same ?

All that getWizPnP (and WizFX and other friends) do is to paste the files together bases on (file, offset, length) triples in the trunc file. If the files haven't been edited on the BW, then this just amounts to concatenation of the files. If VideoRedo just concatenates the files and has a similar problem, then that may be part of it.

[prl]

Version 0.3.2 getWizPnP released

WizPnP device search now works (after a fashion) on Windows and Cygwin. If you run getWizPnP several times in quick succession, the device search can fail. Waithing a minute or two will restore the function again. This appears to be a difference in the way the Windows handles network sockets at the operating system levey. The next version is planned to use caching of search results to work around this problem.

Release notes

New:

• New options --folder and --recursive to select the folder(s) being accessed now also work on local recordings as well as on Beyonwiz recordings.
• Support for WizPnP device search on Windows and Cygwin. But see Bugs.
• Empty path can be given to --indir and --outdir to specify the current directory.

Bugs:

• Uses bignum package for 64-bit integers, even when the underlying Perl integers are 64 bits.
• When resuming a download, may fetch up to 32MB more data than is necessary.
• WizPnP device search is now supported on Linux, MacOS X, Windows and Cygwin. On Windows and cygwin, the search can fail if it is repeated frequently. Waitinh a minute or two clears the problem. This problem will be addressed (probably by caching WizPnP device search results) in the next release.
• Minor known problems that deleting a recording can cause are documented in the BUGS section of the docuentation. It is not known whether deleting a recording using getWizPnP can cause any major problems on the Beyonwiz.
• Does not allow operations on media files from the Beyonwiz contents folder (access to the contents folder is available only in Beyonwiz firmware versions 01.05.261 and later).
• Folder operations only work on firmware 1.05.280 and later
• Sort on title only works on firmware 1.05.280 and later

Fixed:

• More useful error mesage, rather than a Perl internal error when connecting to a Beyonwiz that doesn't have its WixPnP name set.
• More useful error mesage, rather than a Perl internal error when device location information can't be loaded (occurred in some cases where --host and --device were used together).
• Bug in path handling that prevented copy, move and selective listing from working on local recordings now fixed.

[alwayslooking]

Nice to see another update
I'm keen to see how the search works on Windows.
I managed to get a bit of a hack version working yesterday using another utility, but if this works better, I'd prefer to stick with that for WizZilla

[craigh]

Hi Peter

Its all working with the indir option now on Windows XP.

Also it finds the server without Host 2 in 5 times on my setup

Thanks

[prl]

Hi Peter

Its all working with the indir option now on Windows XP.

Also it finds the server without Host 2 in 5 times on my setup

Thanks

Great. I've started working on cacheing the WizPnP searches. That may improve things the reliability.