Series2Folder plugin

[prl]

Series2Folder is a plugin that moves recordings of the same name to folders. It was intended for use with IceTV to get around the fact that it's not possible to specify a recording directory for IceTV series recordings.

The plugin was originally concieved and implemented by BikeMike, and I've been maintaining and improving it with his agreement.

The initial changes I made to BikeMike's code are listed below the more recent release notes.

Using Series2Folder

• Series2Folder moves recordings with the same name into folders that have the same name as the recordings. It can be configured to do that automatically in the background.
• When it has been installed, it is accessed via MEDIA, MENU, and the menu entries "Sel Series to Folder", "Series To Folder" and "Series to Folder...". The three functions can also be assigned to coloured buttons in MEDIA, MENU>Settings. The lengths of these menu names are restricted by the amount of space for them in the coloured buttons.
• "Sel Series to Folder" moves all recordings that have the same name as the currently highlighted recording into a folder. If any recordings are marked, it will operate on those recordings rather than the highlighted recording. It creates the folder if necessary. It will only appear in MEDIA, MENU if it has been enabled in the configuration.
• "Series To Folder" finds all recordings where there are more than a certain (configurable) number of recordings with the same name, and moves them to folders with their respective names, creating the folders if necessary. This action can also be enabled as an automatic background activity. It will only appear in MEDIA, MENU if it has been enabled in the configuration.
• "Series to Folder..." pops up a menu that contains the previous two functions (but with longer and more obvious names), and also an entry to open Series2Folder's setup panel.
• The "Configure move series recordings to folders" settings panel has the following entries:
  • Show move series option: allow "Series To Folder" to appear in the media selection screen menu and/or on coloured buttons. The function is always available in the "Series to Folder..." menu.
  • Show move selected series option: allow "Sel Series To Folder" to appear in the media selection screen menu and/or on coloured buttons. The function is always available in the "Series to Folder..." menu.
  • Automatically create folders: sets the minimum number of recordings with the same name needed before "Series To Folder" will create a directory for them and move recordings into the folder. It applies to both manual and automatic runs of "Series To Folder". Setting it to "no autocreate" means that recordings will only be moved by "Series To Folder" if the folder exists already. Combining "no autocreate" with "Sel Series to Folder" is an easy way to put only some of your series recordings into folders automatically.
  • Strip repeat tags from series names: when enabled, removes a repeat tag from the recording name before using the name as a folder name. When enabled, "Repeat tag to strip" also appears, and allows you to set the repeat tag (defaults to "[R]").
  • Use portable file names: when creating the folder name from the series name convert any characters not allowed in Windows filenames to "_". Most useful if you share the folder with a Windows computer.
  • Put movies into folder: allows Series2Folder to move all recordings whose names start with "Movie: " (case insensitive) into a folder. When enabled, "Folder for movies" appears to allow the name of the folder to be changed (defaults to "Movies").
  • Allow Series to Folder to run in the background: effectively does a "Series To Folder" automatically in the Default movie location shortly after startup and shortly after the end of each recording. The move can be deferred if you are in the media selection screen or in FileCommander.
  • Series to Folder notifications: allows you to control when Series2Folder notifies you about its background actions. The options are "all", 'error and "moved" ' and "error". The "all" option will notify each time Series2Folder runs, even if it finds nothing to do. The default is "error".

Release notes v.1.11

• Add a message to the Series2Folder config popup that tells users how to enable background mode. It shows for a maximum of 5 times, or until background mode is enabled.
• Ask user when cancelling Series2FolderConfig when there are unsaved changes.
• Default to a single Series to Folder... plugin entry in the MEDIA, MENU menu. The menu now no longer includes the "Series to Folder" and "Sel Series to Folder" entries by default. The short names for the Series to Folder and Sel Series to Folder aren't obvious, and are now explained a little in the config popup, where they can be enabled, if desired. If you previously had these entries displayed in the media selection menu, you will need to enable them. This change was made at the request of Colin O'Brien from IceTV.
• Support added for multi-file selection in Beyonwiz MEDIA, MENU screen. You can now mark multiple recordings, and "Move selected series recordings to folder" will move all the recordings matching the marked names to their respective series folders.
• Other minor improvements to explanations.
• Make Series2Folder portable betweewn Python 2 and Python 3 (should be invisible to users).
• Make Series2Folder portable across Beyonwiz, OpenViX and OpenATV (should be invisible to users).
• Make file screen handling more flexible - this helps control when background runs will be blocked by the user being in a context viewing files and where moving files around may casus problems (should be invisible to users).
• Cleanup of code for displaying config items that are only active when other config items are enabled (should be invisible to users).

Release notes v.1.10

• Only show the BLUE Keyboard button hint when focus is on a config item that actually allows text entry.
• Has some changes to make the code more portable to OpenATV and OpenViX, but it is not yet portable to those systems.

Release notes v.1.9

• Allow manual (menu or button) moves of series recordings to folder in any folder, not just in the Default movie location. Adoxa's idea, but a slightly different implementation from what he proposed.
• Background moves now follow the user's setting in Default movie location without having to restart the plugin.

Release notes v.1.8

• Handle all recording filename formats (Long filenames, Short filenames, Event first, as well as standard).
• Avoid over-writing existing files in destination directory.

Release notes v.1.7 (not a separate release, use v1.8 )

• Fix crash when default movie directory doesn't exist.
• Minor code cleanups

Release notes v.1.6

• Fix crash when "zap" timer is running when Series2Folder runs.

Release notes v.1.5

• Fix crash when "Series to Folder is already running in the background." popup was to be displayed.

Release notes v.1.4

• Two minutes after startup, and 5 seconds after the end of each recording, it runs in the background to move any new matching recordings to their destinations.
• It works in a cooperative manner, returning control to the main dispatch loop of the UI frequently to make sure that it doesn't unnecessarily block other operations.
• It is initially disabled, and can be enabled in the plugin's settings. When enabled, you can choose which of the popup notifications you see when it completes a scan: all (including notifications that it has done an automatic scan and not found anything), errors and a list of moved recordings, only errors, or none. The default is "only errors".
• Before it starts and at each processing step, it checks to see whether the system is shutting down or if there is a (or another) recording running and stops its run immediately. If the system stays running, it will run again after the next recording finishes. If the system goes into shutdown, it will run again when it restarts. The "recording running" test may be too conservative, and I'll have a look at whether something better can be done.
• If there are any tasks running or the current screen is either MovieSelection or FileCommander, it defers for a minute (and then retests the conditions). This stops the files being changed "behind the back" of those screens.
• If the manual Series2Folder operations detect that the background Series2Folder is running, they don't do anything, and give an information popup saying that Series2Folder is running in the background.
• Other than the two settings specific to automatic scanning, the automatic scan uses the same configuration settings as the manual scan.

Release notes v.1.3

• Fixed bug that caused a crash if a recording's .ts.meta file couldn't be read.
• Added the version number to the title of the plugin settings popup.

Release notes v.1.2

• Add new PluginDescriptors for single-action "move series to folder" and "move selected series to folder". The single-action plugin descriptors "Series to Folder" and "Sel Series to Folder" have the same effects as the menu entries "Move series recordings to folders" and "Move series recordings to folders" respectively.
• Shortened the plugin descriptors so that they now fit in the button hints if the actions are assigned to coloured button short-presses.
• Split out the "move series" action code into a separate class to allow for the actions, and as a first step to allow automation (should have no user-visible consequences).
• Allow the "Series to Folder" and "Sel Series to Folder" actions to be configured in or out. If configured out, they are still available through the "Series to Folder" popup.
• Avoid moving recordings if they are part of a pending merge for a "save timeshift" recording.
• Don't move any recordings and display a popup if there are jobs running (which could operate on recordings), or if the PVR is in the process of shutting down.
• Allow repeat tagging (e.g. added by IceTV) to be stripped from series names when creating their folders. This also allows repeat movies to be recognised if the repeat tag is at the start of the title (e.g. "[R] Movie: Gone with the Wind"). The default repeat tag is the same as the IceTV default tag, "[R]", but it can be modified to match a different tag configured on IceTV (e.g. '[Rpt]").

Release notes v.1.1

• Optionally (default enabled) replace characters not permitted in Windows filenames (/.\:*?<>|") with '_' when creating folder names from series names.
• When there are errors, or if more than 20 shows are moved to folders, use a TextBox (wider, scrollable and with text wrapping) to display
  the results.
• Use lists to collect user feedback strings and error messages instead of a single string for more flexible presentation.
• When nothing has been moved, increase the timeout on the message box from 5 sec to 10 sec.
• Use gettext for all UI strings.
• Added bitbucket source url to control file.

Original changes to BikeMike's Series2Folder

• Made it a package, so it's a bit easier to install and uninstall (but before installing BikeMike's original version, you have to manually uninstall the original).
• It has a more robust (but a little slower) way of working out the series name than the original.
• When it finishes a run, it automatically updates the list in the Movie Player (MEDIA from live TV).
• It avoids moving recordings that are currently being recorded.
• It has some options, and that means that when you select MENU>Move series recordings into folders, you now get a menu instead of it just running. If you just press OK again (Move series recordings to folders) without making any config changes, you should get the same behaviour as the original - it creates folders for all series with two or more episodes. It then moves the episodes for all series that have folders into their folders, whether the folder was created automatically or not. The folder just has to have the right name.
• There's a new action, Move selected series recording into folder. That acts on just the episodes of the recording that was selected in the Movie Player when you pressed MENU. It will always create a folder for the series if there isn't one, no matter how many episodes there are, and it then moves all episodes into that folder. In combination with a setting below, this allows you to be selective about which series get folders.
• The plugin only appears in the Movie Player's MENU when the focus in the selection list is on a recording or file. This is an inherent limitation of how the menu is constructed in the Movie Player; the plugin can't change that.
• There's a small configuration setup screen with three settings:
  • Automatically create folders: with settings "no autocreate" and "1 recording" .. "10 recordings". This allows you to change the threshold for the number of recordings needed to automatically create a folder in Move series recordings to folders. If it's set to "no autocreate", then Move series recordings to folders will only move recordings to folders that already exist - either created manually, or more conveniently, by Move selected series recording into folder.
  • Put movies into folder: looks for recordings whose names start with "Movie: " and puts them into a folder (defaults to "Movies"). If this is enabled, you can use Move selected series recording into folder to move a single movie into a folder (but Move series recordings to folders will still move all movies into the Movies folder if the Movies folder exists).
  • Folder for movies: the name of the folder to put movies into. If you change this name then movies in the folder with the old name will just stay there, and newly moved movies will go into the folder with the new name.

Installing and updating
If you don't have Series2folder installed or you have an earlier version installed from a package, just install the new version as follows.

On the Beyonwiz, set it to view live TV.

Download the linked enigma2-plugin-extensions-series2folder_version_all.zip file to the top level of a USB stick, unzip it to get the enigma2-plugin-extensions-series2folder_1.0_all.ipk file inside, delete the .zip file and then "Remove safely/Eject" it from your computer and plug it into your Beyonwiz.

After a few seconds you'll get a "The following files were found/Install extensions" popup. Click OK. You should see the .ipk file listed on the screen. Navigate to it and press OK to select it (tick the box). Then press GREEN Install. When that finishes, press EXIT on the "Done" popup, and EXIT again to leave the installer screen.

Then restart the GUI (or reboot).

Once that's done, the Move series recordings into folders entry will be in the MENU list in the Movie Player (MEDIA from live TV).

Uninstalling

Use PLUGIN from live TV, then RED Remove plugin. Navigate to extensions, press OK to open the list, then navigate down to series2folder and press OK. The plugin will be uninstalled. No need to restart the GUI or reboot.

You can't uninstall the original version of Series2folder this way.

Source

The the source code can be found at https://bitbucket.org/prl/series2folder. It's not very big. There's a Makefile, but it depends on copying the plugin to a Beyonwiz to compile it and needs the ipkg-build script (I lifted mine from the opkg-tools package for the Beyonwiz). It's a bit clunky and inconvenient.

The source directory can be downloaded from Bitbucket as a ZIP file and installed in a similar way as the original version if you want to, but it needs to be removed manually before you use the .ipk package. Use the package unless you want to hack the source or you're just nosy

Package downloads

The plugin (already un-zipped) can also be downloaded from Bitbucket at https://bitbucket.org/prl/series2folder/downloads.



.

[Star6key]

Once that's done, the Move series recordings into folders entry will be in the MENU list in the Movie Player (MEDIA from live TV).

The 'cursor' needs to be on a file (not a folder) before you press the Menu key.

[Star6key]

Thanks for this plugin Prl (and BikeMike). Works brilliantly.
I found it useful to assign it to one of the colour keys (in settings), rather than having to go through the menu each time to access it.

[prl]

Once that's done, the Move series recordings into folders entry will be in the MENU list in the Movie Player (MEDIA from live TV).

The 'cursor' needs to be on a file (not a folder) before you press the Menu key.

Yes. That was also the case in the original version. It's true of a number of functions in the MENU list of the Movie Player, not just this one. It also happens to make "Move selected series recording into folder" marginally simpler to code

Specifically:
Reset playback position
Rename
Start offline decode
and all plugins
are left out of the menu if the cursor is on a folder.

Permanently remove all deleted items
is only in the menu if the cursor is on the trashcan.

[prl]

I think I've also thought of a way to allow it to perform updates automatically in the background, too

[MrQuade]

I think I've also thought of a way to allow it to perform updates automatically in the background, too

Heh, I was going to ask if there was some way to trigger the script on finishing a recording or something.
I wait in anticipation . I'll give the plugin a try in it existing form on my T3

[prl]

I think I've also thought of a way to allow it to perform updates automatically in the background, too

Heh, I was going to ask if there was some way to trigger the script on finishing a recording or something.
I wait in anticipation . ...

I was just thinking of just polling regularly, though I could also peek inside the recording timer list and use that to set the timer to trigger just after the end of the next recording, and then drop back to polling if the recording timer list is empty or if the Movie Player is active (testing for that is the bit I haven't tested yet).

Writing this also reminded me to add "It avoids moving recordings that are currently being recorded." to the release notes

[MrQuade]

Oh, I forgot, I should also mention another odd thing.

When the file move had completed, a large popup window was displayed across the entire length of the TV screen and appeared to be wide enough to dissapear off both sides. The popup was not quite the full height of the screen and looked empty, though it might have contained some text off the edge of the screen where I couldn't see it. Pressing Exit dismissed it though.

It's good that the plugin script didn't crash or kill any of my files. It looks like it executed properly. Unfortunately, it looks like the enigma media player can't deal with folders with colons in them. It could display the folder name correctly, and I could enter the directory, just not list the contents.

Not sure what the file commander's problem was. While it could list the files, it appeared to not want to play them.

I just renamed the folder from the command line to omit the colon, and it all appears fine now.

A simple fix might mean that the script just needs a small tweak to omit or substitute any potentially problematic characters.

[Star6key]

Allowing colons (or any other 'Windows' invalid file name characters) is a horrible idea. Sure, they may work in linux, but in Windows... not so much.
On the T4, in Media, I created a folder called ' test: '. In windows, it shows up as ' TFNZPE~P '. This will cause me (and others I'm sure) unnecessary problems.

[Star6key]

As for the limitations of Windows filenames, it's not just colon that will cause you grief!

Correct.

So I did a bit of testing, and here's what I found.

Recording a show with a colon. The fw seems smart enough to change the colons to underscores for the filename (but leaves the colon for display purposes on the T4). e.g. - '20150823 1350 - SBS - Motor Sport: Speedweek.ts' shows as -
20150823 1350 - SBS - Motor Sport_ Speedweek.ts

Changing the recording setting (Setup/TV/Recording settings/Limit character set for recording filenames) to on, changes the same showname to this -
20150823_1350_-_SBS_-_MOTOR_SPORT__SPEEDWEEK.ts (spaces being converted to underscores, is just confusing).

Running the Series2Folder plugin creates a folder with the show name, however, the fw doesn't change any (Windows) invalid characters, so 'Motor Sport: Speedweek' shows up in Windows Explorer as -
MD9WDU~T

Any chance of changing the Plugin to mimic what the fw does for invalid filename characters?

[MrQuade]

The Windows issue should be fine.

Under Samba, the folder gets converted into a bizzare name, but the contents are accessible. I think that side of it is handled as well as it needs to be.

As Stars6ky has pointed out, the filenames are sanitised properly, so it is all handled well.

[prl]

As for the limitations of Windows filenames, it's not just colon that will cause you grief!

Correct.

So I did a bit of testing, and here's what I found.

Recording a show with a colon. The fw seems smart enough to change the colons to underscores for the filename (but leaves the colon for display purposes on the T4). e.g. - '20150823 1350 - SBS - Motor Sport: Speedweek.ts' shows as -
20150823 1350 - SBS - Motor Sport_ Speedweek.ts

It's not really a matter of it "leaving the colon for display purposes". The name displayed is stored in a different place from the file name. This is something that's only done for recordings. The display name for "20151213 1234 - Some Program.ts" is stored, along with other useful information, in "20151213 1234 - Some Program.ts.meta".

For media files, the name displayed is the name in the filesystem; the same for directories.

Changing the recording setting (Setup/TV/Recording settings/Limit character set for recording filenames) to on, changes the same showname to this -
20150823_1350_-_SBS_-_MOTOR_SPORT__SPEEDWEEK.ts (spaces being converted to underscores, is just confusing).

Running the Series2Folder plugin creates a folder with the show name, however, the fw doesn't change any (Windows) invalid characters, so 'Motor Sport: Speedweek' shows up in Windows Explorer as -
MD9WDU~T

Is that the literal name on Windwos?

A file sharing protocol between two dissimilar systems should map names on the server that can't be represented on the client into something that can. If that's really the literal name on Windows, it seems to be doing a particularly poor job.

Any chance of changing the Plugin to mimic what the fw does for invalid filename characters?

"Invalid in Windows" != "Invalid".

I don't think this has anything particular to do with the plugin. Within the GUI I could manually create a directory with the name '*/:"<>?\|' without any restriction, even though they're all illegal long FAT name characters.

It there are to be limits, they should be consistently applied.

What should the restriction be? POSIX Fully portable filenames (A–Z a–z 0–9 . _ -)? ISO 9660 (A–Z 0–9 _ .)? There are lots of different rules, including five different rules for various native Windows file systems.

I think discussion of permitted directory names should be continued as a separate topic in Bug Reporting and Feature Requests.

What undoubtedly does need to change in the plugin is that it should change any '/' characters in series folder names to something else.

[prl]

Oh, I forgot, I should also mention another odd thing.

When the file move had completed, a large popup window was displayed across the entire length of the TV screen and appeared to be wide enough to dissapear off both sides. The popup was not quite the full height of the screen and looked empty, though it might have contained some text off the edge of the screen where I couldn't see it. Pressing Exit dismissed it though.
...

At the end of the operation, Series2Folder displays a summary of what it did in a MessageBox.

MessageBox expands without limit to fit its text (no line wrap).

In the old version, if there were a lot of shows moved, you'd see the list of shows, but it would be cut off top and bottom, which is a bit clumsy, but at least you saw something.

I added information about any errors that happened during the operation to the end of the MessageBox text. If those lines are very long, they can push the list of recordings off the left edge of the screen, and if the list of recordings is long, it can push the error message off the bottom of the screen, so you see nothing.

I think the MessageBox needs to be replaced by a TextBox, which can scroll.

You may be able to get some helpful information about why the error message was so long by trying to move the series one at a time using "Move selected series recording into folder" on each series. That would help me simulate the error (or it may be an error that actually needs fixing!) and make sure the output is tidier.

[prl]

Version 1.1 of Series2Folder plugin available.

[Holden]

I think I've also thought of a way to allow it to perform updates automatically in the background, too

Has this functionality been implemented yet ?
I have the auto create folder set to 1 but it requires me to go in and select move before this happens , it does not automatically create the folder and move the file without user intervention

[prl]

I think I've also thought of a way to allow it to perform updates automatically in the background, too

Has this functionality been implemented yet ?

No, otherwise there'd be a new version with a release note saying it had been implemented in the first post in the topic.

I have the auto create folder set to 1 but it requires me to go in and select move before this happens , it does not automatically create the folder and move the file without user intervention

Yes, that's pretty much what not having an automatic update function means.

[Holden]

I think I've also thought of a way to allow it to perform updates automatically in the background, too

Has this functionality been implemented yet ?

No, otherwise there'd be a new version with a release note saying it had been implemented in the first post in the topic.

I have the auto create folder set to 1 but it requires me to go in and select move before this happens , it does not automatically create the folder and move the file without user intervention

Yes, that's pretty much what not having an automatic update function means.

Thanks for confirming that
It's not just me missing something then

[prl]

There's an issue about how to properly deal with "save timeshift and record to end of show" recordings, where, while the "record to end of show" part is still running, the recording is in two parts, but only one shows up as being recorded. If the plugin runs while this is happening it will probably move the "save timeshift" part, and then when the recording process wants to join the two pieces back together, it won't find the first. "save timeshift" part.

I want to make sure that's dealt with properly before I start automating things.

[prl]

Version 1.2 of Series2Folder plugin available.

Fixes the problem in the previous post and now allows single-button-press (or single menu action) access to SeriesToFolder actions.

The next version will most likely allow for automatic Series2Folder actions.

[MrQuade]

Brilliant work .

Feature wise, this plugin should be ready for inclusion in the standard plugin list I reckon!

[prl]

Thanks.

[IanSav]

Hi,

Feature wise, this plugin should be ready for inclusion in the standard plugin list I reckon!

Rather than bloating the firmware with features that not everyone uses I would rather see a thread created with a recommended or useful plugins list including descriptions and installation instructions. Allow users to choose whether or not they want any particular plugin installed.

This same option should be offered for all the currently hard coded plugins.

Regards,
Ian.

[MrQuade]

Rather than bloating the firmware with features that not everyone uses I would rather see a thread created with a recommended or useful plugins list including descriptions and installation instructions. Allow users to choose whether or not they want any particular plugin installed.

I meant that it should be added to the repository as a downloadable plugin, not as a bundled plugin.

I think it is now useful and stable enough to be an optional feature instead of a thing that you have to download from a forum and install manually.

[IanSav]

Hi MrQuade,

Ahh, that makes more sense and is more than acceptable to address my concerns.

Regards,
Ian.

[prl]

... I meant that it should be added to the repository as a downloadable plugin, not as a bundled plugin. ...

FWIW, I never thought you meant anything else.

[kenmax]

Hi, I like the Plug-in, but I notice it only works on the default Dir.
I have set my Wiz up with a Dir. for wife (default ) and a Dir. for me. It would be handy if the plug-in used the Dir. name it was in as default Dir. to do it's thing, or an option in the settings to do this?
No big deal, it only takes a minute or so to do through the PC anyway, but if it is easy to code and you think it may make the Plug-in more versatile?

[prl]

Hi, I like the Plug-in, but I notice it only works on the default Dir.

That's right.

I have set my Wiz up with a Dir. for wife (default ) and a Dir. for me. It would be handy if the plug-in used the Dir. name it was in as default Dir. to do it's thing, or an option in the settings to do this?

When (in the future) it's running automatically rather than from the Movie Player, what directory is it "in"?

I'll have a think about it and see whether there's anything sensible that can be done.

[simoncasey]

Does that mean you would need to move your files into each folder and then navigate to those folders to run it in there?
I made my own version of the original that checks sub directories as part of the check to see if a folder already exists. Then you can run it from the top level.

[kenmax]

When (in the future) it's running automatically rather than from the Movie Player, what directory is it "in"?

The Plug-in reads the Dir. it has been started from (in) and sets that as the 'Default' Dir.

Does that mean you would need to move your files into each folder and then navigate to those folders to run it in there?

I don't need the missus anywhere near my recordings! So I have root/wife and root/husband, with the recordings going into the appropriate directories.

If I created my Dir. under wife's Dir. does the plug-in do the wife's and sub-dir's?

[tezza007]

So I have root/wife and root/husband...

Nobody? Everybody too PC today?

[prl]

When (in the future) it's running automatically rather than from the Movie Player, what directory is it "in"?

The Plug-in reads the Dir. it has been started from (in) and sets that as the 'Default' Dir. ...

I think you missed the point. When (in the future) it's running automatically rather than from the Movie Player, there is no Movie Player dir to be "in" because it's running automatically and not from the Movie Player. In fact, when it is running automatically, it will check whether the Movie Player is being displayed, and deliberately refrain from doing anything, so that files don't get shifted around from underneath your view of the recordings in the Movie Player.

[prl]

... I made my own version of the original that checks sub directories as part of the check to see if a folder already exists. Then you can run it from the top level.

Would you like to share the code?

I think what you've done might be hard to generalise though, especially in the face of people who put soft links off to NAS directories in their default movies directory.

[Swanny]

Just the plugin I need to put the Midsomer series in.
As a rookie I will let the thread know if I am successful with the installation
I have printed the instructions and downloaded the 1.2 attachment above.

[simoncasey]

... I made my own version of the original that checks sub directories as part of the check to see if a folder already exists. Then you can run it from the top level.

Would you like to share the code?

I think what you've done might be hard to generalise though, especially in the face of people who put soft links off to NAS directories in their default movies directory.

This is what I wanted to do:
I wanted to generate a series of sub directories of the main directory and step through each of those - but my python wasn't up to it so I ended up duplicating the code for a fixed set of directories and specifying each one. For each directory, when it looks for duplicates, it looks in both the subdirectory and in the default movie directory. If it finds a file it gets moved into the series directory. I then run it for the default directory on its own. The only bit is misses is if there is a file in the default directory and a file in the sub directory, that was one step too far.

This is based on Mike's original version, rather than yours.

I also do the thing with files that start Movie:


An extract from the code is here

Code: Select all

    subdir = defaultMoviePath() + '/Horror
    files = [f for f in sorted(os.listdir(rootdir)) if f[0:8].isdigit() and os.path.isfile(os.path.join(rootdir,f))]
    dirs = [f for f in sorted(os.listdir(subdir)) if os.path.isdir(os.path.join(subdir,f))]

    # count number of shows in each series
    counters = {}
    for fullname in files:
        if fullname.endswith('.ts'):
            showname,date_time = recSplit(fullname)
            try:
                counters[showname] += 1
            except KeyError:
                counters[showname] = 1

    # create a directory for each series and move shows into it
    # also add any single shows to existing series directories

    for showname in counters:
        if showname in dirs:
            for fullname in files:
                if (showname + '.') in fullname:
                    if fullname.endswith('.ts'):
                        showname,date_time = recSplit(fullname)
                        moves += '%s - %s\n' %recSplit(fullname)
                    os.renames(os.path.join(rootdir,fullname),os.path.join(subdir,showname,fullname))

    files = [f for f in sorted(os.listdir(subdir)) if f[0:8].isdigit() and os.path.isfile(os.path.join(subdir,f))]
    dirs = [f for f in sorted(os.listdir(subdir)) if os.path.isdir(os.path.join(subdir,f))]

    # count number of shows in each series
    counters = {}
    for fullname in files:
        if fullname.endswith('.ts'):
            showname,date_time = recSplit(fullname)
            try:
                counters[showname] += 1
            except KeyError:
                counters[showname] = 1

    # create a directory for each series and move shows into it
    # also add any single shows to existing series directories

    for showname in counters:
        if counters[showname] > 1 or showname in dirs:
            for fullname in files:
                if (showname + '.') in fullname:
                    if fullname.endswith('.ts'):
                        showname,date_time = recSplit(fullname)
                        moves += '%s - %s\n' %recSplit(fullname)
                    os.renames(os.path.join(subdir,fullname),os.path.join(subdir,showname,fullname))

    # root folder
    subdir = defaultMoviePath() 
    files = [f for f in sorted(os.listdir(rootdir)) if f[0:8].isdigit() and os.path.isfile(os.path.join(rootdir,f))]
    dirs = [f for f in sorted(os.listdir(subdir)) if os.path.isdir(os.path.join(subdir,f))]

    # count number of shows in each series
    counters = {}
    for fullname in files:
        if fullname.endswith('.ts'):
            showname,date_time = recSplit(fullname)
            try:
                counters[showname] += 1
            except KeyError:
                counters[showname] = 1

    # create a directory for each series and move shows into it
    # also add any single shows to existing series directories
    for showname in counters:
        if counters[showname] > 1:
            for fullname in files:
                if (showname + '.') in fullname:
                    if fullname.endswith('.ts'):
                        showname,date_time = recSplit(fullname)
                        moves += '%s - %s\n' %recSplit(fullname)
                    os.renames(os.path.join(rootdir,fullname),os.path.join(subdir,showname,fullname))

If I could do it generically, I would create a list of directories which were the subdirectories in the default movie directory plus the recording directories as per the other thread. I would ignore anything that was a symlink. And then process each one as above but in each folder, you would need to include the recordings directories as well as the default movie location.

A couple of other issues that my code has but you may have fixed. If the movie name includes a - then it gets skipped. Also, if your device stops in the middle of a recording and you end up with two files with the same name and start time, it only moves one of them. I assume they are to do with parsing the filename.

[prl]

... I made my own version of the original that checks sub directories as part of the check to see if a folder already exists. Then you can run it from the top level.

Would you like to share the code?

I think what you've done might be hard to generalise though, especially in the face of people who put soft links off to NAS directories in their default movies directory.

This is what I wanted to do:
I wanted to generate a series of sub directories of the main directory and step through each of those - but my python wasn't up to it so I ended up duplicating the code for a fixed set of directories and specifying each one. For each directory, when it looks for duplicates, it looks in both the subdirectory and in the default movie directory. If it finds a file it gets moved into the series directory. I then run it for the default directory on its own. The only bit is misses is if there is a file in the default directory and a file in the sub directory, that was one step too far.

Thanks for the code.

I think your cut/paste may have truncated the assignment to subDir:

Code: Select all

    subdir = defaultMoviePath() + '/Horror

Is it only missing the closing quote? (And defaultMoviePath() will end with a '/', so it should be just 'Horror' or 'Horror/').

If I could do it generically, I would create a list of directories which were the subdirectories in the default movie directory plus the recording directories as per the other thread. I would ignore anything that was a symlink. And then process each one as above but in each folder, you would need to include the recordings directories as well as the default movie location.

I'm not sure how that would work. How would the code know to put recordings called, say, "Dracula" into "Horror/Dracula", but recordings of, say, "Quantico" into, say, "Thriller/Quantico", unless the series folders were manually created or moved there? And should it only try to put things in folders in direct subdirectories of defaultMoviePath() or anywhere in the subtree under defaultMoviePath()? E.g. should it be able to put "Dracula" into "Horror/Gothic/Dracula"?

A couple of other issues that my code has but you may have fixed. If the movie name includes a - then it gets skipped.

Yes, recSplit() assumes that the show name part of the recording name doesn't contain ' - '.
You can probably fix the problem by changing:

Code: Select all

showname = parts[2].split('.')[0]

to

Code: Select all

showname = parts[2:].join(' - ').split('.')[0]

Also, if your device stops in the middle of a recording and you end up with two files with the same name and start time, it only moves one of them. I assume they are to do with parsing the filename.

recSplit() doesn't deal with the _NNN added to the filename to distinguish what would otherwise be two files with the same filename. One place where this happens is in the circumstances you describe. It happens all over the place if the user selects the short names format for recording files.

My recSplit (which is only called if I can't extract what I want out of the .ts.meta file) is a bit more complicated that the original:

Code: Select all

    def recSplit(self, fullname):
        try:
            startOffset = 2
            parts = fullname.split(' - ')
            date_time = parts[0][6:8] + '.' + parts[0][4:6] + '.' + parts[0][0:4]
            if len(parts[0]) > 8:
                date_time += ' ' + parts[0][9:11] + ':' + parts[0][11:13]
            else:
                startOffset = 1
            showname = splitext(parts[-1])[0]
            if showname[-4:-3] == "_" and showname[-3:].isdigit():
                date_time += '#' + showname[-3:]
                showname = showname[0:-4]
            showname = ' - '.join(parts[startOffset:-1] + [showname])
        except:
            return None, None, False, _("Can't extract show name for: %s") % fullname
        return showname, date_time, False, None

Note that the original only returns (showname, date_time), but you could adapt that to use in your version. My version also handles the short filename format for recordings.

[simoncasey]

I think your cut/paste may have truncated the assignment to subDir:

Code: Select all

    subdir = defaultMoviePath() + '/Horror

Is it only missing the closing quote?

Yes

(And defaultMoviePath() will end with a '/', so it should be just 'Horror' or 'Horror/').

That's what I have and it works, I don't know if it is right.

If I could do it generically, I would create a list of directories which were the subdirectories in the default movie directory plus the recording directories as per the other thread. I would ignore anything that was a symlink. And then process each one as above but in each folder, you would need to include the recordings directories as well as the default movie location.

I'm not sure how that would work. How would the code know to put recordings called, say, "Dracula" into "Horror/Dracula", but recordings of, say, "Quantico" into, say, "Thriller/Quantico", unless the series folders were manually created or moved there? And should it only try to put things in folders in direct subdirectories of defaultMoviePath() or anywhere in the subtree under defaultMoviePath()? E.g. should it be able to put "Dracula" into "Horror/Gothic/Dracula"?

This is what it does for me:
If the series folder exists already, then the file is moved there.
If it doesn't exist already, then it creates the series folder if it finds duplicates. It creates the series folder in the folder that it finds the duplicates as it wont know where else you would want it. So you either have to move the created series folder into the right folder afterwards, or move the files into the right folder before it is run. Once the series folder has been created, it wont matter.
In terms of levels of directory, in theory it should be able to move files into an existing folder whatever sub level it was.

If you end up automating the process, then I imagine it would work like this:
As soon as the second file with the same name is created in the same folder, a new series older will be created and they will be moved into it. This could be from the recording creating the file in the folder or because second file is moved into the same folder.
Once a series folder has been created, any new files of that name get moved into that folder.

Manual intervention will be required if a new series folder has been created and you want it in a specific subfolder or you have individual files that you want to put in a specific subfolder and will be made into series folders later.

[prl]

...

(And defaultMoviePath() will end with a '/', so it should be just 'Horror' or 'Horror/').

That's what I have and it works, I don't know if it is right.
...

It will work, it's just a little untidy (your code will give something like '/media/hdd/movie//Horror'; using 'Horror/' will give '/media/hdd/movie/Horror/' which I also think is untidy, but conforms with the internal convention in enigma2 that directory paths end with '/'. I don't know where that comes from; possibly from someone's aversion to, or ignorance of, os.path.join().

BTW: '/media/hdd/movie//Horror' is interpreted as '/media/hdd/movie/./Horror', '/media/hdd/movie/Horror/' is interpreted as ''/media/hdd/movie/Horror/.'.

[prl]

...
Manual intervention will be required if a new series folder has been created and you want it in a specific subfolder or you have individual files that you want to put in a specific subfolder and will be made into series folders later.

Yes, that's what I was saying.

[simoncasey]

...
Manual intervention will be required if a new series folder has been created and you want it in a specific subfolder or you have individual files that you want to put in a specific subfolder and will be made into series folders later.

Yes, that's what I was saying.

I think that is the best the code could do. Being able to still move files into series folders wherever they have been moved to is the key thing for people who use a folder structure for their recordings.

[prl]

... Being able to still move files into series folders wherever they have been moved to is the key thing for people who use a folder structure for their recordings.

No matter how deep in the directory hierarchy?

I'm not entirely averse to the sort of thing you're suggesting, but I'm very unsure about how it should be implemented.

[simoncasey]

... Being able to still move files into series folders wherever they have been moved to is the key thing for people who use a folder structure for their recordings.

No matter how deep in the directory hierarchy?

I'm not entirely averse to the sort of thing you're suggesting, but I'm very unsure about how it should be implemented.

Ok, I'm not sure how easy that is in python, but if you don't know then it must be hard. Isnt there a walk command in python that can get the directories recursively
You could just assume 1 level of folders which could be genre or person which is the most common.

[prl]

... Being able to still move files into series folders wherever they have been moved to is the key thing for people who use a folder structure for their recordings.

No matter how deep in the directory hierarchy?

I'm not entirely averse to the sort of thing you're suggesting, but I'm very unsure about how it should be implemented.

Ok, I'm not sure how easy that is in python, but if you don't know then it must be hard. Isnt there a walk command in python that can get the directories recursively
You could just assume 1 level of folders which could be genre or person which is the most common.

There is indeed a walk directories method in Python (os.path.walk()), but that's not the "it" I'm referring to: it's the whole issue of how to place series recordings folders anywhere other than directly under a single directory and to automate it in a sensible manner. I don't regard you as a typical user who might want to take advantage of this sort of feature.

[simoncasey]

... Being able to still move files into series folders wherever they have been moved to is the key thing for people who use a folder structure for their recordings.

No matter how deep in the directory hierarchy?

I'm not entirely averse to the sort of thing you're suggesting, but I'm very unsure about how it should be implemented.

Ok, I'm not sure how easy that is in python, but if you don't know then it must be hard. Isnt there a walk command in python that can get the directories recursively
You could just assume 1 level of folders which could be genre or person which is the most common.

There is indeed a walk directories method in Python (os.path.walk()), but that's not the "it" I'm referring to: it's the whole issue of how to place series recordings folders anywhere other than directly under a single directory and to automate it in a sensible manner. I don't regard you as a typical user who might want to take advantage of this sort of feature.

Don't you just create the series folder at the same level you find the multiple files?

[mkwilson]

Series2Folder has all of a sudden started causing a crash when I select 'Sel Series to Folder' (I tried a few different shows...same result). I could see that an error log appear on the screen. Was working fine 3 or 4 days ago. I haven't done any rescans etc (I'm in Canberra, no new channels).... I'm on the beta feed, but don't recall any recent updates.

Could some one kindly point me to the right path such that I can post the error log here?


EDIT: Don't worry....found the log info in the 'Bugs' sub-forum.

[MrQuade]

Crash logs are usually stored here
/home/root/logs

[prl]

The relevant ones will be the ones with the largest numbers in their names. Please post the log as an attachment and I'll try to track down the problem.

[mkwilson]

The relevant ones will be the ones with the largest numbers in their names. Please post the log as an attachment and I'll try to track down the problem.

Thanks Peter. Somewhat strangely, I tried using the plugin this morning.....and it has worked perfectly. Hmmmmm.


EDIT: The log for one of the crashes is attached (hopefully correctly).

[prl]

I've found the bug. Thanks.

[prl]

Version 1.3 of Series2Folder plugin available.

Should fix the bug reported by mkwilson above.

Added plugin version number to the plugin's settinge popup screen.