Yet Another Recording Downloader for the Wiz

[prl]

Assuming you have actually installed VLC (note that VLC is not included with YARDWiz, it is available from VideoLAN), make sure the path to the VLC executable is in your $PATH environment variable. This article suggests it can be done by adding directories that should be in the PATH into files under the /etc/paths.d directory.

Example:
Here is an example of a file that might be called /etc/paths.d/vlc

Code: Select all

/Applications/VLC.app/Contents/MacOS/VLC

Note, I have no idea if this will work as I don't have OSX 10.8 installed.

That seems to be the correct place to add things that should go in everyone's $PATH variable. XQuartz uses it in 10.8, and sets the X11 path in my $PATH variable.

The convention appears to be that the file is named nn-appname, and the nn is used to control the order that the application paths are added to the $PATH variable. The XQuartz one on my system is named /etc/paths.d/40-XQuartz, so a consistent name for VLC might be something like /etc/paths.d/50-VLC.

If VLC is installed in the Applications folder as suggested in the downloaded, that path should be the right contents for /etc/paths.d/50-VLC (you can change the last part of the path to whatever you like and it should still work, but the file must be in the /etc/paths.d folder).

Don't bother trying to find /etc/paths.d in the Finder. It's not normally visible there. But it is visible from the command-line interface in Terminal. TextEdit can't access the directory directly, either.

[DavidR]

Yes VLC is installed and working.
Tried the fix above and it still doesn't work.

[prl]

I'll download YARDWiz and have a look at it in the morning. I'm running Mountain Lion 10.8.2.

[prl]

Hi, Luke. I just noticed that you publish an SHA1 checksum for the OSX YARDWiz DMG.

Unfortunately, OS X doesn't appear to come with a command-line SHA1 command. MD5, yes, nut not SHA1.

[prl]

Yes VLC is installed and working.
Tried the fix above and it still doesn't work.

You need to log out and log back in before the fix can work. Apologies if that's obvious to you and you've already done it.

[DavidR]

Ditto, also on 10.8.2, upgrade from 10.7.

Actually did a reboot rather than just a logout - still no good.
Should I at least see the VLC icon in YardWiz?

The path in 50-VLC is /Applications/VLC.app/Contents/MacOS/VLC.

[Luke]

Hi, Luke. I just noticed that you publish an SHA1 checksum for the OSX YARDWiz DMG.

Unfortunately, OS X doesn't appear to come with a command-line SHA1 command. MD5, yes, nut not SHA1.

Google code generates and publishes the SHA1checksums automatically and doesn't have an option to use MD5 (though I could generate them manually and insert them into the download description text field).

[Luke]

Should I at least see the VLC icon in YardWiz?

You'll only see the icon if YARDWiz can locate VLC, it's hidden otherwise. Will VLC start if you type "vlc" in a command prompt?

[DavidR]

Yes - if I type "open -a vlc" at a bash prompt.

[prl]

Hi, Luke. I just noticed that you publish an SHA1 checksum for the OSX YARDWiz DMG.

Unfortunately, OS X doesn't appear to come with a command-line SHA1 command. MD5, yes, nut not SHA1.

Google code generates and publishes the SHA1checksums automatically and doesn't have an option to use MD5 (though I could generate them manually and insert them into the download description text field).

I suspected something like the SHA1 checksum being generated automatically. I probably wouldn't have bothered with checking an MD5 checksum if there had been one there, though.

If I'd been hacked sufficiently to point me to a fake YARDWiz page containing malware, the hackers probably would have also been smart enough to post checksums that correspond to the malware I'd be downloading.

[prl]

OK, I've installed YARDWiz (0.4.2.1-OSX-10.6) and VLC (2.0.5) in /Applications, and put the path /Applications/VLC.app/Contents/MacOS in /etc/paths.d/50-VLC.

I'm seeing (or rather not seeing) the same thing as DavidR; there's no VLC icon at the bottom of the queue window and no Play in VLC entry in the context menu for a recording in the recordings list.

The VLC directory is listed when I print the PATH variable in Terminal, and I can run VLC just by typing the command VLC in a Terminal window, so the correct path is being loaded into the shell run by Terminal.

However, it looks as though neither the paths in /etc/paths nor the paths in /etc/paths.d are in the environment that YARDWiz uses. Logging out and back in after adding the VLC information to /etc/paths.d doesn't seem to help.

The YARDWiz log file says that YARDWiz's PATH is:

Code: Select all

DEBUG __init__._ApplyConfig: PATH
        /Applications/YARDWiz.app/Contents/Resources
        .
        /usr/bin
        /bin
        /usr/sbin
        /sbin

That path doesn't contain either the VLC or (on my system) XQuartz paths that are in /etc/paths.d. It also doesn't contain /usr/local/bin, which is listed in /etc/paths.

Perhaps YARDWiz could add the paths in those files to its PATH variable, and additionally (as the documentation says it does for Windows) add the usual installation location for VLC to the PATH as it does for the YARDWiz resources.

I'm happy to help with any testing. I owe Luke a lot for his testing of getWizPnP.

Speaking of which, Luke, do you think that getWizPnP 0.5.4beta1 is ready for release?

[DavidR]

EDIT: spoke to soon. I've got the VLC icon appearing, but it won't launch VLC.

Fixed (I think...) Had to create the file /etc/launchd.conf and add the line

setenv PATH /Applications/VLC.app/Contents/MacOS/:/usr/bin:/bin:/usr/sbin:/sbin

You'd need to add anything else in your existing path that's not default.

It could also be that the line we were adding to paths and /paths.d/vlc is /Applications/VLC.app/Contents/MacOS/VLC and not /Applications/VLC.app/Contents/MacOS - we were specifying the app and not its path.

See
http://apple.stackexchange.com/question ... at-will-be

EDIT: see this about editing info.plist for the app in question http://apple.stackexchange.com/question ... 9845#79845

[prl]

EDIT: spoke to soon. I've got the VLC icon appearing, but it won't launch VLC.

Fixed (I think...) Had to create the file /etc/launchd.conf and add the line

setenv PATH /Applications/VLC.app/Contents/MacOS/:/usr/bin:/bin:/usr/sbin:/sbin

You'd need to add anything else in your existing path that's not default.

I've now done that, and I get the context menu Play in VLC and the VLC icon (required reboot, not just logout to do it). And it didn't work for me either. If I select Play VLC in the context menu for a recording, nothing happens. If I start a download and then click on the VLC icon at the bottom of the queue, I get an error message in the Log window saying:

Code: Select all

Downloading Test/Seven Morning News...
Finished playing recording
vlc: unknown option or missing mandatory argument `--qt-minimal-view'
Try `vlc --help' for more information.

I don't appear to get this error message if I use the context window to play (but it doesn't work, either).

When I run vlc --help, there is no --qt-minimal-view (and indeed no other --qt-*) option listed.

If I go into YARDWiz Tools>Options...>Settings and remove the --qt-minimal-view option from VLC commandline arguments, then VLC runs for both the context menu and the VLC icon playing a recording as it downloads.

...
It could also be that the line we were adding to paths and /paths.d/vlc is /Applications/VLC.app/Contents/MacOS/VLC and not /Applications/VLC.app/Contents/MacOS - we were specifying the app and not its path. ...

It could be that that's what you were doing, but it wasn't what I was doing. I was able to use /etc/paths.d/50-VLC to let me run VLC from the Terminal without specifying a path.

The problem with /etc/paths and /etc/paths.d isn't that I was getting the path wrong (I tested that), but that those files don't appear to be used to construct PATH in the environment that YARDWiz has. As I said in my previous post.

[prl]

Some additional information, mostly negative, about setting up the PATH in launchd using launchctl commands:

• Although the man page for launchctl implies that ~/.launchd.conf can be used to configure launchd in a per-user way, the man page for launchd.conf contradicts that. The man page for launchd.conf is correct.
• Unfortunately launchctl can't expand environmental variables, so you can't do something like "setenv PATH $PATH:/Applications/VLC.app/Contents/MacOS" in /etc/launchd.conf to just add to the path launchd already has.
• launchd starts as the first process at boot time, and it only reads /etc/launchd.config at that time.
• If you want to experiment with paths in launchd, you can run something like "launchctl setenv PATH /some:/new/path"
• Messing about with system configuration files in Terminal is probably not something the average YARDWiz users will want to do.

[Luke]

• Messing about with system configuration files in Terminal is probably not something the average YARDWiz users will want to do.

Agreed, I'll add the VLC path to YARDWiz, the same as what I do for Windows users.

[prl]

Hi, Luke.

Can I make a suggestion about the PATH that is set up by YARDWiz for itself? The one I saw in the log files has the current directory '.' earlier in the path than the normal system command directories like /bin and /usr/bin.

The current directory, if it appears at all in $PATH, should be after all the system directories in $PATH. This prevents some (fairly simple-minded) security attacks.

The current directory shouldn't be in YARDWiz's PATH unless it's actually necessary, and if it is there it should be late in the list, after all the system command directories.

[prl]

• Messing about with system configuration files in Terminal is probably not something the average YARDWiz users will want to do.

Agreed, I'll add the VLC path to YARDWiz, the same as what I do for Windows users.

Sounds good

[Luke]

Speaking of which, Luke, do you think that getWizPnP 0.5.4beta1 is ready for release?

Haven't really had a chance to test well, but the major issue from 0.5.3 (resuming) seems to be resolved. I'll have another go at testing over the weekend.

I've uploaded a YARDWiz beta with VLC fixes (removed --qt-minimalview arg and added OSX specific VLC path) and moved '.' to the end of the PATH. I'd appreciate it if someone could test on OSX 10.8 (I've still only got a 10.6.7 'hackintosh' virtual machine which is a pain to upgrade).

http://code.google.com/p/yardwiz/downlo ... X-10.6.dmg

As an aside, testing the compiled version of getWizPnP 0.5.4beta1 on OSX the first time produced the following error, possibly because the existing unpacked par in $TEMPDIR was 0.5.2. Removing it as per the error message resolved the issue.

getWizPnP: private subdirectory /var/folders/I4/I4wbd3buF8Crh3zsvPa-UE+++TI/-Tmp-//par-bnl is unsafe (please remove it and retry your operation)

[prl]

Speaking of which, Luke, do you think that getWizPnP 0.5.4beta1 is ready for release?

Haven't really had a chance to test well, but the major issue from 0.5.3 (resuming) seems to be resolved. I'll have another go at testing over the weekend.

I've uploaded a YARDWiz beta with VLC fixes (removed --qt-minimalview arg and added OSX specific VLC path) and moved '.' to the end of the PATH. I'd appreciate it if someone could test on OSX 10.8 (I've still only got a 10.6.7 'hackintosh' virtual machine which is a pain to upgrade).
http://code.google.com/p/yardwiz/downlo ... X-10.6.dmg

Thanks, Luke. I'll download the YARDWiz beta and give it a spin.

As an aside, testing the compiled version of getWizPnP 0.5.4beta1 on OSX the first time produced the following error, possibly because the existing unpacked par in $TEMPDIR was 0.5.2. Removing it as per the error message resolved the issue.

getWizPnP: private subdirectory /var/folders/I4/I4wbd3buF8Crh3zsvPa-UE+++TI/-Tmp-//par-bnl is unsafe (please remove it and retry your operation)

Perhaps a new version of par/pp sets the permissions on $TMPDIR/par-username tighter than previous versions or checks more carefully. When I first run getWizPnP 0.5.4beta1, the par cache has Unix permissions 700. But I get the same permissions when I run 0.5.3 after deleting the old par cache, and no error if I then run 0.5.4beta1. However, if I delete the par cache, run 0.5.3, manually change the permissions on the par cache to 755, and then run 0.5.4beta1, I can force the error. However, if I then run 0.5.2, I get no error. If I delete the par cache and run 0.5.2 it creates the cache with 755 permissions.

So it appears that the permissions on the par cache have been tightened up, but I don't recall seeing this error message.

[Luke]

I'll download the YARDWiz beta and give it a spin.

Don't bother, found a bug that didn't show up on my OSX VM as it doesn't have VLC installed... Sorry if you've already downloaded beta1.

[Luke]

Righto, beta2 uploaded, this time after installing VLC on my OSX VM. YARDWiz finds VLC OK, but files won't actually play for me. Though I think that's a VM issue as virtualbox doesn't expose my graphics card to OSX and VLC complains about requiring OpenGL.

http://code.google.com/p/yardwiz/downlo ... X-10.6.dmg

[prl]

I've downloaded it. When I went to download, it said it was a deprecated version. I assume that that's just a convention you use to keep betas out of your main download list.

I had Default to TS Format for Downloads set for all the tests.

It seems to be pretty much random whether VLC starts playing the recording immediately, or it just shows the VLC control panel and I have to start play manually.

Each time VLC starts in a YARDWiz session, an extra --no-video-title gets added to the argument list, so it can end up lookilg like this in ps:

Code: Select all

 1671   ??  S      0:00.78 vlc --extraintf=rc --rc-host=localhost:51440 --quiet --verbose=0 --rc-fake-tty --deinterlace-mode=mean --no-video-title --no-video-title --no-video-title --no-video-title --no-video-title /var/folders/nm/mzgkvw5j06bg3c4rvfvqc8dm0000gn/T/tmpNwCRLh.ts

Sort of repetitively redundantly tautological. Harmless, but untidy (though only geeks who know how to use ps will ever know).

I just don't seem to be able to get some recordings to play in VLC.

When I try to play a non-playable recording using Play in VLC in the recording download window, getWizPnP runs (I can see it in a ps listing), but VLC doesn't (or at least it exits/crashes before I get to look at it in ps). There's no error message in the log file. When it's in this state, Play in VLC is greyed out in the context menu, and if I kill getWizPnP from a Terminal window the log file shows:

Code: Select all

Error, unable to download /var/folders/nm/mzgkvw5j06bg3c4rvfvqc8dm0000gn/T/tmpcb5JqJ.ts.
getWizPnP STDOUT:PRIME Canberra: Better Homes And Gardens/The 80's Revisited - Deborah Conway, Jon Stevens and Richard Clapton / Piano Bar / The Rubik's Cube Bedroom / Ant Farms and Sea Monkeys / Crepe Suzette - Copy
getWizPnP STDERR:

If I kill the getWizPnP process, Play in VLC returns to active in the context menu.

If I start downloading a non-playable recording (i.e. one that Play In VLC won't play), the VLC icon at the bottom of the window remains greyed out.

I haven't seen anything peculiar in the info window for any of the recordings that don't play. Here's one:

Code: Select all

PRIME Canberra: Better Homes And Gardens/The 80's Revisited - Deborah Conway, Jon Stevens and Richard Clapton / Piano Bar / The Rubik's Cube Bedroom / Ant Farms and Sea Monkeys / Crepe Suzette
Ant farms and sea monkeys suggest a bizarre cast of mind; so too,
perhaps a Rubik's Cube bedroom. Or the fashions of the eighties, not
so the music though, Deborah Conway, Jon Stevens and Richard Clapton
take us down memory lane. And crepe suzette and piano bars a good in
any decade. Or century.
Index name: recordings/Test/Better Homes And Gardens Jul.23.2010_19.28
Fri Jul 23 19:28:00 2010 - Fri Jul 23 20:45:00 2010
playtime:   77:00    recording size:   3099.1 MB    bit rate:   5.4 Mb/s
autoDelete: Never

An interesting thing about this is that I generated a bunch of edited copies of it to test various parts of getWizPnP's downloading of edited recordings, and none of the edits of the original plays, either.

That non-playable one was on PRIME Canberra, but none of the old copies I have of Wallace and Gromit's World of Invention that I've tried to play, play either. Here's the info fro one of them:

Code: Select all

ABC1: Wallace And Gromit's World Of Invention/From A To B
In the last episode of the series, Wallace and Gromit explore
incredible transportation inventions from around the globe as the
desire for freedom and escape have driven human inventiveness through
the ages. In the segment 'Contraption Countdown' the focus is on
peculiar transport devices and we find out how to cross the River
Thames in a plastic bag.
Index name: recordings/Wallace & Grommit/Wallace And Gromit's World Of Invention Feb.27.2011_18.28
Sun Feb 27 18:28:00 2011 - Sun Feb 27 19:25:00 2011
playtime:   57:00    recording size:   2239.6 MB    bit rate:   5.2 Mb/s
autoDelete: Never

All of the recordings I've tried are SD. I don't currently have any HD ones on that Beyonwiz. Some other PRIME Canberra recordings play. None of the ABC1 recordings I tried played. Could it be that you're seeing this prlblem when you try to run VLC, rather than VLC failing completely?

Final item is a suggestion: Why separate pause and resume download buttons? Why not swap the function and icon by context, rather than greying out the non-appliccable one?

Sorry that it's got a bit long-winded, but I wanted to get everything down that I'd tried.

[prl]

I've enabled debugging and had a look at the debug log output for a recording that plays successfully from Play in VLC, and one that doesn't.

Here's what's in the log for one that plays:

Code: Select all

DEBUG utilities.subproc: getWizPnP --version
DEBUG utilities.subproc: ['getWizPnP', '--version']
DEBUG utilities.__init__: /var/folders/nm/mzgkvw5j06bg3c4rvfvqc8dm0000gn/T/tmpJRruIt.ts
DEBUG utilities.subproc: getWizPnP --all -q -R --BWName -O /var/folders/nm/mzgkvw5j06bg3c4rvfvqc8dm0000gn/T -T tmpJRruIt "recordings/Test/Escape To The Sun Jan.9.2011_22.43" -H 10.1.1.8 -p 49152 -t
DEBUG utilities.subproc: ['getWizPnP', '--all', '-q', '-R', '--BWName', '-O', '/var/folders/nm/mzgkvw5j06bg3c4rvfvqc8dm0000gn/T', '-T', 'tmpJRruIt', 'recordings/Test/Escape To The Sun Jan.9.2011_22.43', '-H', '10.1.1.8', '-p', '49152', '-t']
DEBUG utilities.subproc: vlc --extraintf=rc --rc-host=localhost:49435 --quiet --verbose=0 --rc-fake-tty --deinterlace-mode=mean --no-video-title /var/folders/nm/mzgkvw5j06bg3c4rvfvqc8dm0000gn/T/tmpJRruIt.ts
DEBUG utilities.subproc: ['vlc', '--extraintf=rc', '--rc-host=localhost:49435', '--quiet', '--verbose=0', '--rc-fake-tty', '--deinterlace-mode=mean', '--no-video-title', '/var/folders/nm/mzgkvw5j06bg3c4rvfvqc8dm0000gn/T/tmpJRruIt.ts']

For one that doesn't play I see:

Code: Select all

DEBUG utilities.subproc: getWizPnP --version
DEBUG utilities.subproc: ['getWizPnP', '--version']
DEBUG utilities.__init__: /var/folders/nm/mzgkvw5j06bg3c4rvfvqc8dm0000gn/T/tmpDa9duu.ts
DEBUG utilities.subproc: getWizPnP --all -q -R --BWName -O /var/folders/nm/mzgkvw5j06bg3c4rvfvqc8dm0000gn/T -T tmpDa9duu "recordings/Test/Better Homes And Gardens Jul.23.2010_19.28" -H 10.1.1.8 -p 49152 -t
DEBUG utilities.subproc: ['getWizPnP', '--all', '-q', '-R', '--BWName', '-O', '/var/folders/nm/mzgkvw5j06bg3c4rvfvqc8dm0000gn/T', '-T', 'tmpDa9duu', 'recordings/Test/Better Homes And Gardens Jul.23.2010_19.28', '-H', '10.1.1.8', '-p', '49152', '-t']

That's not truncated. there's no mention of VLC at all, so it seems that utilities.subproc isn't being called for VLC.

I also noted another oddity. When I run ps to look at the processes that YARDWiz is forking, getWizPnP shows up like this:

Code: Select all

  482   ??  S      0:09.30 ./getWizP --all -q -R --BWName -O /var/folders/nm/mzgkvw5j06bg3c4rvfvqc8dm0000gn/T -T tmpDa9duu recordings/Test/Better Homes And Gardens Jul.23.2010_19.28 -H 10.1.1.8 -p 49152 -t

It looks like argv[0] is getting truncated to 9 characters. Perhaps there's a copy of 9 chars from the executable path name that used to work when the name was "getWizPnP", but now truncates, because "./getWizPnP" is 11 chars, and so argv[0] is being truncated to "./getWizP", and the "nP" is being chopped off. I'm not sure whether this is happening in YARDWiz code or in the Python libraries, but the fact that it's specific to the length of the string "getWizPnP", points more to it being in YARDWiz.

[Luke]

Each time VLC starts in a YARDWiz session, an extra --no-video-title gets added to the argument list

Fixed.

Final item is a suggestion: Why separate pause and resume download buttons? Why not swap the function and icon by context, rather than greying out the non-appliccable one?

That's similar to how I originally implemented it. I was using two buttons in the same spot and showing/hiding as required which worked fine on Windows but didn't work correctly on Ubuntu so I was lazy and just put the two buttons next to each other. I've just rewritten to use a single button and switch icon/tooltip/function as required. This works on Ubuntu, should have done this in the first place...

Not sure if I can debug the VLC issues in OSX, but I'll have a look.

[prl]

What really puzzles me is why the problem I'm seeing depends on the recording.

I'm not sure I should be suggesting this, but I could have a play with the 0.4.3beta2 source.

My Python's a bit rusty*, though (and never was all that great).

* Now that's a bizarre image!

[Luke]

What really puzzles me is why the problem I'm seeing depends on the recording.

I'm not sure I should be suggesting this, but I could have a play with the 0.4.3beta2 source.

Feel free to play with the code, it is open-source after all (note: I'm not a programmer, have no programming training and only taught myself python to automate data analysis tasks at work). Have you tried ruling out VLC itself by downloading then playing directly in VLC (not via YARDWiz)? What version of VLC? VLC 2.x seems to be a major rewrite, I had better luck with 1.1.9 - http://download.videolan.org/pub/videol ... 1.9/macosx

[prl]

What really puzzles me is why the problem I'm seeing depends on the recording.

I'm not sure I should be suggesting this, but I could have a play with the 0.4.3beta2 source.

Feel free to play with the code, it is open-source after all (note: I'm not a programmer, have no programming training and only taught myself python to automate data analysis tasks at work). Have you tried ruling out VLC itself by downloading then playing directly in VLC (not via YARDWiz)? What version of VLC? VLC 2.x seems to be a major rewrite, I had better luck with 1.1.9 - http://download.videolan.org/pub/videol ... 1.9/macosx

From what I can see from the logs, the problem isn't that VLC won't play the recordings, but that YARDWiz doesn't even start VLC.

[Luke]

Ah... I completely missed your post this morning

[Luke]

I also noted another oddity. When I run ps to look at the processes that YARDWiz is forking, getWizPnP shows up like this:

Code: Select all

  482   ??  S      0:09.30 ./getWizP --all -q -R --BWName -O /var/folders/nm/mzgkvw5j06bg3c4rvfvqc8dm0000gn/T -T tmpDa9duu recordings/Test/Better Homes And Gardens Jul.23.2010_19.28 -H 10.1.1.8 -p 49152 -t

It looks like argv[0] is getting truncated to 9 characters. Perhaps there's a copy of 9 chars from the executable path name that used to work when the name was "getWizPnP", but now truncates, because "./getWizPnP" is 11 chars, and so argv[0] is being truncated to "./getWizP", and the "nP" is being chopped off. I'm not sure whether this is happening in YARDWiz code or in the Python libraries, but the fact that it's specific to the length of the string "getWizPnP", points more to it being in YARDWiz.

I think that's just OSX chopping off the last 2 characters in the terminal as getWizPnP still runs correctly. From my last test:

Code: Select all

bnls-Mac:~ bnl$ ps -e|grep -i wiz
  272 ??         0:35.51 /Users/bnl/YARDWiz-0.4.3/dist/YARDWiz.app/Contents/MacOS/YARDWiz -psn_0_114716
  353 ??         0:02.08 ./getWizP --all -q -R --BWName -O /var/folders/I4/I4wbd3buF8Crh3zsvPa-UE+++TI/-Tmp- -T tmpD7gTst recordings/Motor Sport_ Speedweek Jan.13.2013_11.27 -H 192.168.0.4 -p 49152 -t
  355 ttys000    0:00.01 grep -i wiz
bnls-Mac:~ bnl$ ps -e|grep -i vlc
  356 ??         0:00.76 vlc --extraintf=rc --rc-host=localhost:49521 --quiet --verbose=0 --rc-fake-tty --deinterlace-mode=mean --no-video-title /var/folders/I4/I4wbd3buF8Crh3zsvPa-UE+++TI/-Tmp-/tmpD7gTst.ts
  359 ttys000    0:00.03 grep -i vlc

[prl]

I also noted another oddity. When I run ps to look at the processes that YARDWiz is forking, getWizPnP shows up like this:

Code: Select all

  482   ??  S      0:09.30 ./getWizP --all -q -R --BWName -O /var/folders/nm/mzgkvw5j06bg3c4rvfvqc8dm0000gn/T -T tmpDa9duu recordings/Test/Better Homes And Gardens Jul.23.2010_19.28 -H 10.1.1.8 -p 49152 -t

It looks like argv[0] is getting truncated to 9 characters. Perhaps there's a copy of 9 chars from the executable path name that used to work when the name was "getWizPnP", but now truncates, because "./getWizPnP" is 11 chars, and so argv[0] is being truncated to "./getWizP", and the "nP" is being chopped off. I'm not sure whether this is happening in YARDWiz code or in the Python libraries, but the fact that it's specific to the length of the string "getWizPnP", points more to it being in YARDWiz.

I think that's just OSX chopping off the last 2 characters in the terminal as getWizPnP still runs correctly. From my last test:

Code: Select all

bnls-Mac:~ bnl$ ps -e|grep -i wiz
  272 ??         0:35.51 /Users/bnl/YARDWiz-0.4.3/dist/YARDWiz.app/Contents/MacOS/YARDWiz -psn_0_114716
  353 ??         0:02.08 ./getWizP --all -q -R --BWName -O /var/folders/I4/I4wbd3buF8Crh3zsvPa-UE+++TI/-Tmp- -T tmpD7gTst recordings/Motor Sport_ Speedweek Jan.13.2013_11.27 -H 192.168.0.4 -p 49152 -t
  355 ttys000    0:00.01 grep -i wiz
bnls-Mac:~ bnl$ ps -e|grep -i vlc
  356 ??         0:00.76 vlc --extraintf=rc --rc-host=localhost:49521 --quiet --verbose=0 --rc-fake-tty --deinterlace-mode=mean --no-video-title /var/folders/I4/I4wbd3buF8Crh3zsvPa-UE+++TI/-Tmp-/tmpD7gTst.ts
  359 ttys000    0:00.03 grep -i vlc

Argv[0] is just a piece of text passed to the exec() or execv() system calls. It doesn't necessarily say anything about what was passed as the name of the executable to run (that's a separate argument to exec[v]). However, the convention is that argv[0] is the name of the executable.

It's perfectly possible to run something like exec("/bin/ls", "not_ls_at_all", NULL), and it will show up in ps as "not_ls_at_all". It's also possible for an executable to modify its own argument list, though that's unusual.

The fact that getWizPnP runs means that the executable path name was correct. The fact that YARDWiz's argv[0] shows up as /Users/bnl/YARDWiz-0.4.3/dist/YARDWiz.app/Contents/MacOS/YARDWiz strongly suggests that it's not an OS limitation that's causing argv[0] to be chopped off for ./getWizPnP.

What do you see in ps if you run ./getWizPnP from the Terminal?

[Luke]

What do you see in ps if you run ./getWizPnP from the Terminal?

Code: Select all

bnls-Mac:Resources bnl$ ./getWizPnP 
ABC1: 7.30 (ACT) *LOCKED
etc…
bnls-Mac:Resources bnl$ echo $PATH
/Library/Frameworks/Python.framework/Versions/2.6/bin:/Library/Frameworks/Python.framework/Versions/2.7/bin:/usr/bin:/
bnls-Mac:Resources bnl$ getWizPnP 
-bash: getWizPnP: command not found
bnls-Mac:Resources bnl$ PATH=$PATH:.
bnls-Mac:Resources bnl$ getWizPnP
ABC1: 7.30 (ACT) *LOCKED
etc…

From another terminal:

Code: Select all

bnls-Mac:~ bnl$ ps -e|grep -i wiz|grep -v grep
  453 ttys000    0:01.21 ./getWizPnP
bnls-Mac:~ bnl$ ps -e|grep -i wiz|grep -v grep
  458 ttys000    0:01.44 ./getWizP

[prl]

I've been able to repeat it.

If I don't have '.' in my path, and I run ./getWizPnP, then I see:

Code: Select all

Cambyses:~ prl$ ps -e | grep -i w\\iz
 1807 ttys000    0:01.23 ./getWizPnP
Cambyses:~ prl$

If I put '.' at the start of my path (temporarily, of course ), and I run getWizPnP, I see:

Code: Select all

Cambyses:~ prl$ ps -e | grep -i w\\iz
 1813 ttys000    0:01.22 ./getWizP
Cambyses:~ prl$

Why the grep works as it does is left as an exercise for the reader

It's weird. If I try the same thing with a real binary like sleep, no such truncation happens. So it appears that it's not a problem in the shell, but could be a problem with the startup code in the packed version of getWizPnP.

Just as an aside, here's a perl one-liner to demonstrate the disconnect between the executable name and argv[0].

Code: Select all

Cambyses:getWizPnP prl$ perl -e '$prog="/bin/sleep"; exec $prog "not_sleeping_just_napping", "30"' &
[1] 1876
Cambyses:getWizPnP prl$ ps -e | grep slee\\p
 1876 ttys000    0:00.01 not_sleeping_just_napping 30
Cambyses:getWizPnP prl$ 

[Luke]

Interesting, though I think it's a bit of a red herring in regards to some of your TS files not playing.

I think it's worth checking whether VLC will actually play them once completely downloaded by getWizPnP and I'll put in some extra debugging code to see if I can find out what's going on with YARDWiz.

[prl]

I tried to build the YARDWiz source, but:

• I don't have python 2.6, I have python 2.7.
• I don't have a python binary anywhere in /Library/Frameworks, only in /usr/bin.
• When I try to manually emulate what's in the build-yardwiz-osx.sh script by copying getWizPnP into yardwiz-read-only, and then running "python setup.py py2app", the build proceeds to make a YARDWiz app and DMG in the dist directory, but I get a whole bunch of errors from the strip command: "/usr/bin/strip: changes being made to the file will invalidate the code signature in: /Volumes/Cambyses_User/Users/prl/Computer/src/Beyonwiz/yardwiz-read-only/dist/YARDWiz.app/Contents/Resources/lib/python2.7/lib-dynload/fcntl.so (for architecture x86_64)" which makes me uneasy about the build.
• The uneasiness is confirmed when I try to run the built YARDWiz app, and I get a pop-up window just saying "YARDWiz error" with an option to open the console. When I do, there is just a console message saying "Dock: no information back from LS about running process", and then "YARDWiz: YARDWiz error".

Have I missed something in the build process? Do I have to have python 2.6? Are the error messages from strip significant?

I'll download one of my "unplayable" recordings and see if I can play it directly in VLC.

[Luke]

You don't need to build, just run python yardwiz or ./yardwiz from the source directory. The script just needs to be able to find getWizPnP.pl/getWizPnP in your PATH.

Edit: actually, it might not be that straightforward as I assume you probably don't have the wxpython python library installed.

That build script is just a helper for my specific environment. The simplest way to run would be to copy any updated files to an existing YARDWiz.app folder structure (not sure of the exact path as I'm at work now) they're just stored as .py/.pyc files so you can just replace them.

[prl]

Curiouser and curiouser...

When I downloaded one of the "unplayable" (via the Play in VLC menu item) recordings, the download starts, but there's no progress shown in the progress bar. The log shows:

Code: Select all

Downloading Test/BHG Tail1...
Error, unable to download /Volumes/Cambyses_User/Users/prl/Desktop/BHG Tail1 2010-07-23.ts.
getWizPnP STDOUT:PRIME Canberra: BHG Tail1/The 80's Revisited - Deborah Conway, Jon Stevens and Richard Clapton / Piano Bar / The Rubik's Cube Bedroom / Ant Farms and Sea Monkeys / Crepe Suzette - Copy
getWizPnP STDERR:
Retrying (attempt 2).
Downloading Test/BHG Tail1...
Error, unable to download /Volumes/Cambyses_User/Users/prl/Desktop/BHG Tail1 2010-07-23.ts.
getWizPnP STDOUT:PRIME Canberra: BHG Tail1/The 80's Revisited - Deborah Conway, Jon Stevens and Richard Clapton / Piano Bar / The Rubik's Cube Bedroom / Ant Farms and Sea Monkeys / Crepe Suzette - Copy
getWizPnP STDERR:Recording BHG Tail1 2010-07-23 - The 80's Revisited - Deborah Conway, Jon Stevens and Richard Clapton - Piano Bar - The Rubik's Cube Bedroom - Ant Farms and Sea Monkeys - Crepe Suzette.ts already exists
Use --force to overwrite it
Copy failed: Forbidden
Retrying (attempt 3).
Downloading Test/BHG Tail1...
Error, unable to download /Volumes/Cambyses_User/Users/prl/Desktop/BHG Tail1 2010-07-23.ts.
getWizPnP STDOUT:PRIME Canberra: BHG Tail1/The 80's Revisited - Deborah Conway, Jon Stevens and Richard Clapton / Piano Bar / The Rubik's Cube Bedroom / Ant Farms and Sea Monkeys / Crepe Suzette - Copy
getWizPnP STDERR:Recording BHG Tail1 2010-07-23 - The 80's Revisited - Deborah Conway, Jon Stevens and Richard Clapton - Piano Bar - The Rubik's Cube Bedroom - Ant Farms and Sea Monkeys - Crepe Suzette.ts already exists
Use --force to overwrite it
Copy failed: Forbidden

However, the downloaded file appears to be complete and plays just fine in VLC.

Strangely, if I download the same recording using either the plain Perl version of getWizPnP (0.5.4beta1) or the packed version of getWizPnP included in YARDWiz 0.4.3beta2, I don't get the errors and the downloaded recording TS file is byte-for-byte identical with the one that getWizPnP run from YARDWiz says didn't download.

Even stranger is that even if I manually run the getWizPnP in YARDWiz with exactly the same arguments that YARDWiz uses, I don't get any errors either (and the downloaded recording is byte-for-byte identical with the other downloads).

If I download a "playable" recording, the progress bar works and I get no download errors:

Code: Select all

Downloading Test/Seven Morning News...
Download of /Volumes/Cambyses_User/Users/prl/Desktop/Seven Morning News 2012-02-17.ts complete.

[prl]

You don't need to build, just run python yardwiz or ./yardwiz from the source directory. The script just needs to be able to find getWizPnP.pl/getWizPnP in your PATH.

Edit: actually, it might not be that straightforward as I assume you probably don't have the wxpython python library installed.

That build script is just a helper for my specific environment. The simplest way to run would be to copy any updated files to an existing YARDWiz.app folder structure (not sure of the exact path as I'm at work now) they're just stored as .py/.pyc files so you can just replace them.

You're right. I don't have wxpython installed. That can easily be corrected, I imagine. What version should I get?

I've also discovered that I do have python2.6 installed. It's just not in /Library/Frameworks/Python.framework where your script has it, it's in /System/Library/Frameworks/Python.framework. I guess that this is either an OS X 10.6/10.8 difference or an XCode version difference. I'm not sure whether python comes with the standard distribution of OS X or is in XCode.

[Luke]

wxPython2.8-osx-unicode-py2.6 or wxPython2.8-osx-unicode-py2.7 from http://wxpython.org/download.php
Python 2.6 is not required, that's just what I've got installed on the OSX VM. YARDWiz runs fine using Python 2.7.

[prl]

OK, I've found the cause of the problem I'm having with both Play in VLC and some downloads flagging errors for no apparent reason.

The problem arises because I have $episode (the perl variable corresponding to the --episode option) set in my config file. This means that getWizPnP appends the episode name (if there is one) to the downloaded file name. YARDWiz is unaware of this, and when it checks to see that the file exists, or tries to find the size of the file, it fails, because the file is being downloaded to a different name from what YARDWiz expects. Hence the "download failed" error and the lack of the progress bar.

I haven't looked at the code that runs VLC, but I assume that it waits for the recording file to be present before it starts VLC, which, because it's looking for the wrong filename, never happens.

This is why all the recordings I've looked at have either both problems or neither problem. The problem occurs if a recording has an episode name set, which is why some recordings were OK, and some failed like this.

A workaround for this is to change like 543 of yardwizgui/utilities.py from:

Code: Select all

cmd=[wizexe,'--all','-q','-R','--BWName','-O',d,'-T',f,program['index']]+self.device.args

to

Code: Select all

cmd=[wizexe,'--all','-q','-R','--BWName','-O',d,'--noepisode','--nodate','--nodateLast','-T',f,program['index']]+self.device.args

I think that that's untidy, and it leaves open other strange problems arising from side-effects of things that a user might put in their .getwizpnp/getwizpnp.conf file.

Possible better fixes to the problem are:

• An option to getWizPnP to ignore the user config file and run with the normal coded defaults.
• An option to allow the specification of an alternative config file.

The latter is more flexible, and would allow YARDWiz to specify its own config file options if desired (or just point getWizPnP at an empty config file).

However, I think I can now report that YARDWiz playback using VLC does work on 10.8.2.

[Luke]

Hahaha, I should have thought that you of all people might have a custom config file... - http://code.google.com/p/yardwiz/issues/detail?id=2

[prl]

Hahaha, I should have thought that you of all people might have a custom config file... - http://code.google.com/p/yardwiz/issues/detail?id=2

How would you prefer to address the problem?

[Luke]

I do like the idea of allowing the specification of an alternative config file. However, I can also work around by temporarily modifying %APPDATA% / $HOME environment vars at runtime.

[prl]

I do like the idea of allowing the specification of an alternative config file. However, I can also work around by temporarily modifying %APPDATA% / $HOME environment vars at runtime.

messing about with the environment doesn't sound a great way to do it.

On reflection, I think I can combine both my fixes in getWizPnP into a single option (it has far too many already), so I think I'll go that way.

Expect a new beta sometime this week.

[prl]

The config file is read (actually executed, it's just perl code) before the command-line options are decoded (config settings override program defaults; command line arguments override both). So no option to set the config file without a fair bit of re-coding of how options are processed.

I have made a modification to have environment variable GETWIZPNPCONF set the config file.

E.g. on Bourne-style shells (like bash), you can do this:

Code: Select all

$ export GETWIZPNPCONF=~/mygetwizpnp.conf
$ getWizPnP ... [uses new conf file]

or, for a one-off run with a different conf file:

Code: Select all

$  GETWIZPNPCONF=~/mygetwizpnp.conf getWizPnP ... [uses new conf file]

Not as convenient to code as an option, but will that do?

[Luke]

Yes, that's easy to handle. I can specify the environment each subprocess executes in without messing with the overall environment.

[prl]

getWizPnP 0.4.5 beta2 now available for download. Adds the ability to set the getWizPnP config file name by setting the GETWIZPNPCONF environment variable.

[bpratt]

Just installed this under win7, and it has a problem when I try to save configs, with the see the logfile.. catch is the logfile does not exist.

c:\users\brendan\appdata\local\temp\yardwiz.20130209.205938.9709.log


where and what is the config file, and can it be edited in notepad ??

Sorry if the answer is already in a post in this 30 page thread, but couldn't seem to find a easy way to search for the answer.

[Luke]

There's a bug in the current release that crashes saving preferences when VLC is not installed. Workaround is to install VLC or edit %APPDATA%\yardwiz\config.ini in notepad.

[prl]

There's a bug in the current release that crashes saving preferences when VLC is not installed. Workaround is to install VLC or edit %APPDATA%\yardwiz\config.ini in notepad.

That explains what I saw, then. YARDWiz was crashing for me when I saved preferences, and then suddenly it stopped crashing, so I wasn't able to reproduce the problem. I suspect that installing VLC was the reason that the crashes stopped.