Improvement request. Media display. Directory contents

[prl]

...
I may think about contributing in some fashion, where is the source?
...

The body of the UI source for the Beyonwiz T series is in the Beyonwiz Bitbucket repository: https://bitbucket.org/beyonwiz/. The UI code is in https://bitbucket.org/beyonwiz/easy-ui-4. The issue tracker is in https://bitbucket.org/beyonwiz/easy-ui-4/issues. There are 100+ open issues, so there's plenty to chose from

The UI code is in two main parts: the Python upper layer, and a C++ lower layer. I mostly work on the Python code, but I also have the (Linux) toolchain to build the C++ part. You can modify and test the Python code simply by copying the Python files to the right place in the PVR firmware (/usr/lib/enigma2/python).

It's fairly big - about 145000 lines each respectively of Python and C++.

There are presently only three users who are making contributions to improving the code, so each additional person makes a relatively large difference.

BTW: I'd actually like to have directory contents counts in the media browser. It's just that I haven't found a good way to implement it. My preference would be to have counts of playable files and subdirectories on directory entry, but making that distinction requires a stat(2) system call and file system access for each entry in each subdirectory, which makes it a good bit slower than mechanisms that only look at the file name text.

[peteru]

+1 for the feature itself. However, the implementation must be reliable and perform well in all cases, including very large and deep directories on network volumes. That includes the dreaded DLNA servers too.

[prl]

I don't believe that "deep" is a problem for this feature (by which I mean listing counts for immediate subdirectories), because the depth accessed in any one screen is fixed. But "large" is.

[dRdoS7]

Hi,

I, and I can also speak for my wife (& caravan, which now has it's own T2 ), don't see any need to know how many files are in a directory.

So that's -3.

dRdoS7

[paullings]

After starting to use Series2Folder I think displaying the number of files in a folder would be an essential feature. If it's possible it would be displayed when highlighting the folder.
So +1 from me if it's possible.

[prl]

There's been plenty of support for the idea for a long time. Even the ones who grumble about implementation, like peteru and me, think it would be a good thing to have.

[adoxa]

Here's my approach (source; script attached). There's no configuration yet, but to avoid potentially lengthy operations, each subdirectory has a one-second timeout. It replaces the "Directory" text with the directory (if any) and file (if any) counts (e.g.: "Directories: 1; Files: 2", "Directories: 1", "Files: 2"); empty (and special) directories remain as "Directory".

[prl]

Any hang in scanning a directory through the servicefs system will hang at list(), not at getNext(). I think that the hang is also likely to block the thread.

[adoxa]

Ah, I see. In that case, a quick scan will have to wait, as I can't compile, yet. However, I've setup the configuration, so now counting can be disabled (enabled is default) and individual subdirectories can be disabled via the menu (creating ".e2_do_not_count" in the directory).

[prl]

Ah, I see. In that case, a quick scan will have to wait, as I can't compile, yet. ...

I should have checked the code first. The scan of a servicefs directory doesn't happen in serviceHandler.list(), but rather in the first call of reflist.getNext(). But I don't think that makes much difference to the problem with the timeout in your count() method. See
eServiceFS::getContent(std::list<eServiceReference> &list, bool sorted)
and
eServiceFS::getNext(eServiceReference &ptr)
in lib/service/servicefs.cpp

I guess it's to avoid holding a opendir() file descriptor open across multiple calls and the need for the higher level code to close it if the list isn't exhausted by getNext() calls. Possibly similar considerations are behind the fact that the Python os.listdir() returns a list rather than an iterator.

[adoxa]

FWIW I've updated my directory counter to update the count when adding a file to the directory or toggling its status. It will also only display "Directory" if the directory isn't counted, using "No matches" if a filter is in effect, "Empty" otherwise (empty as in nothing of interest, rather than physically empty).

[paullings]

Hi Adoxa,
I've found a problem with subdir scan on my U4. Whenever I reboot the file count disappears. I then reinstall it however it is still installed as the install message says it has been removed. Reinstall and it works. Reboot and the file count disappears.

[adoxa]

I don't know why that would happen. Here's the new version (as dir-count/"Directory count" now, but the same thing). Note that this is incompatible with multiple selections, so it's one or the other, atm.

[paullings]

Aaaah. I had both installed. I have now installed the new directory count and all seems good. Thanks for the new improved version.
I hope your enhancements end up being accepted into the firmware.

[adoxa]

With the lack of response I hadn't submitted a pull request, but have done so now.

[kenmax]

Can someone point me in the right direction on how to install the .zip by adoxa please

[Paul_oz53]

Can someone point me in the right direction on how to install the .zip by adoxa please

The general process is to: expand the zip file; copy the files to a USB stick; and then, insert the USB stick in the Wiz.

If it finds the .sh file it will ask if you want to install it. If it does, answer yes. (Not sure if this is the default or not - I do it this way so rarely).

Otherwise, press Menu>Sources/Files. Select the USB stick from the set of folders available. Select the .sh file to install.

[kenmax]

Thanks, worked a treat!

[dRdoS7]

Hi,

Can someone point me in the right direction on how to install the .zip by adoxa please

The general process is to: expand the zip file; copy the files to a USB stick; and then, insert the USB stick in the Wiz.

If you can't be bothered to use USB, it's possible to do it over a LAN. Copy the unzipped files to a folder on a drive that's mounted (or create a new one), navigate via "Sources/Files", and run the ".sh". I usually copy the files to a dir on the T2 so it can be installed again.

dRdoS7.

[Paul_oz53]

Hi,

Can someone point me in the right direction on how to install the .zip by adoxa please

The general process is to: expand the zip file; copy the files to a USB stick; and then, insert the USB stick in the Wiz.

If you can't be bothered to use USB, it's possible to do it over a LAN. Copy the unzipped files to a folder on a drive that's mounted (or create a new one), navigate via "Sources/Files", and run the ".sh". I usually copy the files to a dir on the T2 so it can be installed again.

dRdoS7.

+1 - That's what I usually do.

I pitched my reply at a basic level to avoid having to explain how to set up a LAN if kenmax hadn't done so already.

[grampus]

Ahh, all good things come to he who waits.
request completed.
Thanks.

[adoxa]

Here's an update that should (as yet untested) allow directories outside of home to be counted (once enabled). It also includes marking.

[Paul_oz53]

Here's an update that should (as yet untested) allow directories outside of home to be counted (once enabled). It also includes marking.

I'll give this a try over the weekend and let you know how it goes.

Thanks,
Paul

[paullings]

Here's an update that should (as yet untested) allow directories outside of home to be counted (once enabled). It also includes marking.

Many thanks Adoxa for the quick response to my query.
I can't seem to get this version to work though. I uninstalled the plugin version on both my U4 and T3 and then installed this new version on both. It works on both but I can't get it to work on the T3 when viewed as a network mount from the U4.

[adoxa]

I thought I had a newer version, but forgot about the plugin. Here's that updated and tested.

[paullings]

I thought I had a newer version, but forgot about the plugin. Here's that updated and tested.

Thanks Adoxa. No problems here and most importantly dear wife is very happy.

[rfmoore]

I have downloaded the attachment below to a usb drive and can see it on the T4 but do now know how to install it. Can you advise the appropriate steps needed.
enigma2-plugin-systemplugins-directorycount_20180729.2_all.ipk.tgz
(57.15 KiB)

Reg

[adoxa]

Installation instructions here.

[000]

Thank you plugin author + posters. I grabbed the downloads, put them in place and got it to install nicely (2nd attempt).

For the "future" wish-list: It would be good if we didn't have to "tag" the directories we want the file-count on...

But a small price to play for a function that my wife and I both have desperately wanted! Once 'tagged' the folders do what they're meant to do / do what the plugin says 'on the box'

Thanks again!

D.

[MrQuade]

If the tags weren't there you'd run the risk of the Wiz bogging down to a crawl as it tries to count everything it sees.

[adoxa]

For the "future" wish-list: It would be good if we didn't have to "tag" the directories we want the file-count on...

Open the movie list, press MENU, select "Settings..." and turn on "Show directory counts" - everything under the default movie directory (typically /media/hdd/movie) will be counted by default (then you can "tag" directories to not count). You'll still need to explicitly turn counting on for directories outside of that path (that's what this update is for, to allow counting on a network share).

[000]

Edit (found the option):

Hi adoxa,

Ah, thank you, found that now on the 2nd page of Settings (have to learn to scroll more!)

Thank you,
D.

[prl]

have to learn to scroll more!

In most screens that can scroll, the scroll bar is present whenever scrolling is possible.

[Paul_oz53]

Installed the plugin version easily.

Works fine on the T4 and I can also select the T3 directories from the T4 and enable counting on specific folders too.

I'll update the U4 shortly but not expecting any differences to the T4 experience. Version 1 is working fine on it.

A very nice enhancement. Thanks Adoxa.

[adoxa]

It's been merged, so V2 beta users will have it now (20190608), but others will have to keep using the plugin for a few months more.

[paullings]

It's been merged, so V2 beta users will have it now (20190608), but others will have to keep using the plugin for a few months more.

That's great it's now been merged into the firmware.
Just out of interest do you know why the T and U series users will have to wait a few months for an update.

[adoxa]

Just out of interest do you know why the T and U series users will have to wait a few months for an update.

That's the estimate peteru gave for when he'll update them from 17.5 to 19.3. When things settle down a bit more I might put out an Experimental that includes all the non-binary changes.

[prl]

Just out of interest do you know why the T and U series users will have to wait a few months for an update.

That's the estimate peteru gave for when he'll update them from 17.5 to 19.3.

To expand a little - the version number change is the version number of the system components underlying the UI code - build tools, the firmware OS kernel, libraries, firmware support programs, etc, etc. That can bring out all sorts of niggly issues that take time to resolve. As I understand it, peteru also doesn't have much time to dedicate to that effort at the moment.

Hence the delay.

[prl]

I have a speedup to the directory counts currently on test in our on-use T4. It improves the time to do the counts considerably when there are many directories and/or many files in the directories.

The speedup comes from only calling serviceHandler.info(serviceref) when it is necessary: when counting the number of subdirectories and when tag filtering is enabled. It may be possible to avoid it when counting subdirectories, too.

To implement it required changing the test for OS X junk ._* files from using serviceHandler.info(serviceref).getName(serviceref) for the test to using the path name in the serviceref. IMO this is more correct anyway.

If it seems to work correctly, I'll post a patch script to allow others to have a try.

[prl]

I've created Enhancement request #736: Speed up MovieSelection directory counts.

Comments there or here welcome.

[prl]

This is alpha firmware and is not intended for normal use.

Here's a alpha patch that implements my speedups for subdirectory file/directory counts in the media selection list. I've been running it on our in-use T4 for a while now. It should run on any model running firmware 20191106. It was developed and initially tested on the V2.

If you don't use the directory count feature, it should make no difference to the load time for the media selection list.

Amongst other things, it makes a slight change (that I think is actually more correct that the original code) to the skipping of MacOS files that start with "._".

The patch has been tested with firmware version 20191106, and it should also work in 20200328.

To apply the patches, download the linked .ZIP file, and extract it. It will create a new directory/folder called dircount-installer, which contains two files, installer.sh and uninstaller.sh.

Copy the two files somewhere convenient on a T/U/V series box (like /home/root), then log into the box using telnet or ssh, change directory to the place you put the installer.sh/uninstaller.sh files. If you put the files in /home/root you'll be in the right place as soon as you log in.

To install the patches run:

sh installer.sh

and restart the GUI (or reboot).

To uninstall the patches, log in and go to the directory as you did to install, and run

sh uninstaller.sh

And restart the GUI.

Make sure you uninstall before doing an online upgrade.

You can check whether the patches are installed by logging in and running this on the box:

find /usr/lib/enigma2 /usr/bin /usr/share/enigma2 -name \*.bak

It should print nothing if the patch isn't installed, and it should print

/usr/lib/enigma2/python/Components/MovieList.pyo-dircount.bak

if the patch is installed.

The patch can also be installed by running the scripts from FileCommander (MENU>sources / Files from live TV). Doing it that way avoids the need to use the command-line interface and have a login set up on the PVR. It also allows you to run the install/uninstall scripts from a USB thumb drive, and avoids the need to have the box networked (though I suspect that most folk who will use this have it networked anyway).

Comments welcome!

[adoxa]

Not sure how much of a difference it would make, but couldn't a directory (mustDescent) also use serviceref.getPath() directly? And the normal load could probably test ._ before getting the name.

[paullings]

Hi PRL,
On my U4 with a mount to my T3 this is noticeably quicker.

Thanks,

[prl]

Not sure how much of a difference it would make, but couldn't a directory (mustDescent) also use serviceref.getPath() directly?

If the directory type is idFile, getName() is fast (essentially the same as doing getPath()) and if the type is something else, getPath() is necessary, so I left it as getPath(). It is something I thought about

And for recordings it's actually the serviceHandler.info(serviceref) part of the name fetch that bears the cost, not the getName() call.

And the normal load could probably test ._ before getting the name.

That's a good idea, but it's the serviceHandler.info() that needs to be moved (and duplicated as is done in count().

[prl]

Hi PRL,
On my U4 with a mount to my T3 this is noticeably quicker.

Great to hear

[prl]

And the normal load could probably test ._ before getting the name.

That's a good idea, but it's the serviceHandler.info() that needs to be moved

Currently testing this.

(and duplicated as is done in count().

It turns out that wasn't necessary.

[prl]

I've submitted the speedups to directory counts to the firmware source repository, including adoxa's suggestion (that's not in the alpha) about moving the test on "._" in MovieList.load().

The alpha patch should work in firmware 20200328 as well as in 20191106.

[prl]

The speedups have been merged into the firmware source repository. They should be in the next firmware update.

[grampus]

Can anyone make a comment on a limit to how many entries can be shown as a directory count.

E.G.
I have a number of directories with 2 - 3 shows, with the number count shown on the right side of the directory

I have one directory that has currently 139 files
yet the media list just gives
Show name "spicks and specks" and "directory" with nothing appended
All others list show name "Files" and the appropriate number. or "empty"
Not a big deal, But just wonder if there is a limit?