FileCommander.Inputmod & InputBoxmod

[IanSav]

Hi Prl,

I don't know if or when I installed the unrar package but it is on my machine. (it is available as opkg install unrar.) I may have installed it as part of my working with creating packages.

Regards,
Ian.

[prl]

I'm sure I remember unrar being part of the standard install. But I'll just catch the exception (in all the FileCommander archive unpack classes).

[prl]

While I was digging around in the code for FileCommander cleaning up the imports, I got interested in FileCommander/addons/dmnapi.py and dmnapim.py, which didn't seem to be used in the code.

They appear to be an interface to the napiprojekt.pl API for fetching subtitles for media files based on an MD5 checksum of the first 10MB of the file.

It looks as though they might have been hand-copied from somewhere, rather than file copied, because there are some obvious typos:

Code: Select all

            if fps < 22 < 32:
                ...
            if not 22 < fps < 32:

where the first test should almost certainly be the inverse of the second test (i.e. if 22 < fps < 32:), and:

Code: Select all

    try:
        fps = float([re.match(r".*<fps>(.*)</fps>.*", x).groups(0)[0] for x in urllib2.urlopen(url) if x.find('<fps>') > 0][0])
    except:
        fps = 23.976
    return floatfps

where floatfps isn't assigned to (and float(fps) is unnecessary).

Does anyone know about the origin of the code (there are hints that it originated on the dreambox in the code)? Is napiprojekt.pl a good source of subtitles? Is the lookup method (hash on the first 10MB of the file) robust enough to be useful? Is there a better database to use instead?

The one media file I have for testing, an MP4 of an episode of Proof, returns Polish subtitles when requested, but not English ones.

It would be useful for me to get a reasonably-sized sample of MD5 hashes of media files to test out to see what the English coverage is like for subtitles in napiprojekt.pl. I've attached a script to generate the hashes. The simplest way to run it is to copy it to /home/root on the PVR, and then something like:

Code: Select all

chmod a+x ~/getNapiHashes.sh
cd /path/to/media/files
~/getNapiHashes.sh *.mp4

E.g.:

Code: Select all

root@beyonwizu4:~# cd /media/hdd/movie
root@beyonwizu4:/media/hdd/movie# ~/getNapiHashes.sh *.mp4
d41d8cd98f00b204e9800998ecf8427e  - linkfile.mp4
02568829e0a0bbefd5e51d814726002b  - proof.us.107.hdtv-lol.mp4
d41d8cd98f00b204e9800998ecf8427e  - real.mp4
root@beyonwizu4:/media/hdd/movie#

You can add more specific filename expansion to the arguments if there are a large number of files.

It's probably not useful to supply hashes for more than one episode of a series.

[glen]

Given the current push to ban subtitle websites I am not sure that there is any point.

https://www.technadu.com/australian-ent ... tes/37194/

Assuming their push succeeds then access to subtitles will be significantly reduced. Yes, it is very easy to get around the Australian content filtering but it is really worth the effort?

[prl]

I'm not sure what the legal status is of the subtitle database on napiprojekt.pl. They appear to sell a product to access the database. napiprojekt.pl isn't on the list of targeted subtitle sites, but I guess that doesn't mean they won't be at some stage.

The Python files I found appear to have been an enigma2 plugin at some point (the files contain code to update themselves from an opkg feed). There doesn't seem to be anything in the enigma2-plugins repository, though.

[glen]

This is the list of sites they are attempting to block currently, no doubt it will grow in the future

https://torrentfreak.com/images/siteblock.pdf (referenced from this story https://torrentfreak.com/movie-tv-show- ... er-180802/)

napiprojekt.pl doesn't appear to be on it but most of the common subtitle web sites are on it unfortunately, including the one that I always use.

[prl]

This is the list of sites they are attempting to block currently, no doubt it will grow in the future

The court blocks by individual IP addresses? Do copyright owners need to go back to court each time the provider switches IP address (but keeps the DNS name)?

[glen]

My understanding is that they block the URL at the ISP DNS servers. That is why the filtering is so easy to get around.

[prl]

Steel doors in paper walls...

[glen]

Yeah, it is pretty useless but it makes everyone look like they are doing something about it.

Anyway from my perspective it is better than them implementing something that is effective

[adoxa]

While I was digging around in the code for FileCommander cleaning up the imports, I got interested in FileCommander/addons/dmnapi.py and dmnapim.py, which didn't seem to be used in the code.
[...]
Does anyone know about the origin of the code (there are hints that it originated on the dreambox in the code)? [...]

Looks like the plugin is available here (Polish, Google translation; last release appears to be 2015) and the source is here (last update five months ago).

[prl]

Thanks.

[prl]

Implementations of:
Enhancement #673: Tidy up FileCommander list headers
Enhancement #681: Show link targets in File Commander file status
Enhancement #685: Show link "arrows" on files as well as directories in File Commander

are in firmware 20180809beta, but there is a bug in #681 that means that it only shows the correct link information when used on links in /home/root (where I tested it ).

Edit: bug fix submitted.

[peteru]

Edit: bug fix submitted.

Unfortunately, to the wrong repository.

[prl]

Edit: bug fix submitted.

Unfortunately, to the wrong repository.

Fixed.

It's really annoying when I do that.

Is it possible to change the default?

[IanSav]

Hi Prl,

Do you intend to keep the new FileCommander heading skin variables list_left_head1 and list_right_head1 or will you revert to the original names list_left_head and list_right_head when the coding is complete? For the time being I have coded OverlayHD to support both versions with a conditional attribute but I wanted to get some insight into your plans.

My preference would be to keep the original names for the final version of the code.

Regards,
Ian.

[prl]

Do you intend to keep the new FileCommander heading skin variables list_left_head1 and list_right_head1 or will you revert to the original names list_left_head and list_right_head when the coding is complete? For the time being I have coded OverlayHD to support both versions with a conditional attribute but I wanted to get some insight into your plans.

My preference would be to keep the original names for the final version of the code.

As far as I'm concerned, that code is finished. Neither of the two header parts perform the same function as the old heading. It was mainly for that reason that the old name wasn't retained.

[IanSav]

Hi Prl,

Okay. As soon as the next beta ships I will pull the original names. Thanks.

Regards,
Ian.

[IanSav]

Hi Prl,

Actually, after thinking on this a bit more. The code functionality has changed but the new functionality should have no bearing on the variable names used in the skin. I think it would be better and more appropriate to get rid of the "1" in the list_left_head1 and list_right_head1 names. When I see numbered variables like these I go looking for the other variables in the set. In this case there are none.

Regards,
Ian.

[prl]

Actually, after thinking on this a bit more. The code functionality has changed but the new functionality should have no bearing on the variable names used in the skin. I think it would be better and more appropriate to get rid of the "1" in the list_left_head1 and list_right_head1 names.

In the old code & OverlayHD skin, a single text field held filename + file status. In the new code, that is split into two fields, a text field that holds the file name and a templated field that holds the file status. In the old OverlayHD skin, that was covered by a single 3-line widget. The replacement for that to match the new code is a 2-line widget to hold the file name and a single-line widget to hold the file status. The two arrangements occupy the same amount of space at the top of the file list.

I don't think there is any clean way to have the skin cover both arrangements without using 3 different widget variable names, whatever they are.

When I see numbered variables like these I go looking for the other variables in the set. In this case there are none.

I don't understand. There is more than one variable in the set:

Code: Select all

		self["list_left_head1"] = Label(path_left)
		self["list_left_head2"] = List()
		self["list_right_head1"] = Label(path_right)
		self["list_right_head2"] = List()

The code for that is the same in both FileCommanderScreen and FileCommanderScreenFileSelect.

I suggest that OverlayHD skin this for backward compatibility as:

Code: Select all

		<widget name="list_left_head" conditional="list_left_head" position="50,80" size="580,60" backgroundColor="TextBackground" font="Regular;18" foregroundColor="Text" transparent="1" valign="center" />
		<widget name="list_left_head1" conditional="list_left_head1" position="50,80" size="580,40" backgroundColor="TextBackground" font="Regular;18" foregroundColor="Text" transparent="1" valign="center" />
		<widget source="list_left_head2" conditional="list_left_head2" render="Listbox" position="50,120" size="580,20" backgroundColor="TextBackground" foregroundColor="Text" selectionDisabled="1" transparent="1" >
			<convert type="TemplatedMultiContent">
				{"template": [
					MultiContentEntryText(pos = (0, 0), size = (115, 20), font = 0, flags = RT_HALIGN_LEFT, text = 1), # index 1 is a symbolic mode
					MultiContentEntryText(pos = (130, 0), size = (90, 20), font = 0, flags = RT_HALIGN_RIGHT, text = 11), # index 11 is the scaled size
					MultiContentEntryText(pos = (235, 0), size = (260, 20), font = 0, flags = RT_HALIGN_LEFT, text = 13), # index 13 is the modification time
					],
					"fonts": [gFont("Regular", 18)],
					"itemHeight": 20,
					"selectionEnabled": False
				}
			</convert>
		</widget>

[IanSav]

Hi Prl,

I apologise for my earlier posts. I was wrong. Your explanation post put me back on the right track. I am working on too many things at the same time and it appears I have temporarily lost the plot.

Regards,
Ian.

[IanSav]

Hi Prl,

OverlayHD has been updated to use your suggested changes. Thank you.

Regards,
Ian.

[prl]

That's fine. I'm glad we're back on the same page. I was a bit puzzled along the way, though.

[IanSav]

Hi Prl,

I am integrating the new VirtualKeyBoard into OpenPLi and there are a number of unexpected issues. They have resulted in code crashes that only occur in OpenPLi. It turns out that they do skins in a different way and their Directories.py does not have some of the smarts in the OpenViX and Beyonwiz versions.

It is difficult to balance development across three different Enigma2 images. As in this case I got a little lost.

Regards,
Ian.

[prl]

The bug in:
Enhancement #681: Show link targets in File Commander file status
that caused the link target to not be displayed has been fixed in firmware 20180821.

20180821 also cleans up import statements: replaces import * statements with explicit imports from the module, removes unused imports and replaces things like from os.path import isdir as os_path_isdir with import os; os.path.isdir(). It's possible that something was missed in doing those changes, so watch out for crashes caused by things not being imported when they should have been.

Adoxa has also made some improvements to FileCommander that are in 20180821:

[FileCommander] Preserve index on move/delete

When FileCommander's file list updates with a file name it would move to the top, which is where it would stay if the file is no longer present (after move or delete). Prevent that, so the list remains at the same relative location.

[FileCommander] Use functions, not commands

File Commander would delete and rename single files using shell commands rm and mv. Use the functions remove and rename, instead. Use a console container to remove a single directory, not a console screen.

[FileCommander] Copy/move selected movie parts

File Commander would not copy or move associated movie files (such as *.meta) when they were multiselected. Only skip them when the movie itself is also selected.

[adoxa]

Use a console container to remove a single directory, not a console screen.

(Heh, it's prl's post, but really my quote.)

This change seems to require a manual refresh, yet not via multiselect; maybe the loop overhead gives it just enough time (for an empty directory, anyway). In any event, I soon hope to add automatic refresh when a move or copy job, or directory delete, is completed, along with adding "(n jobs)" to the title. Only for the current FC jobs, though, not in general and not if you exit and come back (and the job title won't happen during multiselection, either).

[prl]

Yes, you probably need to do the refresh in the callback from the console container.

[adoxa]

Here's a patch to try out the new automatic refresh, to see if anything needs changing before I submit it. It also includes the rename fix, as well as enabling 8 for mediainfo, and allows files to be copied/moved to the root (which will be part of the submission). Source included, or check it out here.

[adoxa]

Any objections if I change 5 from "Default Folder" to "Location..."? It would open a menu with a list of bookmarks and mounts. Bookmarks would be created/removed via a menu entry (nothing automatic). I was thinking of creating a default list comprising the default path (if set), /etc/enigma2, /home/root, /usr/share/enigma2 and the default movie path. The order would be changed via PREV & NEXT.

[IanSav]

Hi Adoxa,

Are you sure that adding all those system directories is a smart idea? Users who need to go there will get there fast enough. Leading non technical users to these locations is likely to end in tears.

I don't think PREV or NEXT are appropriate selections for moving an item up or down one entry. I think LEFT and RIGHT for move up and down one line. PREV and NEXT could be for move to first line and move to last line. These have the same scopes as the other uses of these buttons.

Regards,
Ian.

[adoxa]

Are you sure that adding all those system directories is a smart idea? Users who need to go there will get there fast enough. Leading non technical users to these locations is likely to end in tears.

How many non-technical users use File Commander? Still, a fair point, so just default to the existing default path and default movie path?

I don't think PREV or NEXT are appropriate selections for moving an item up or down one entry. I think LEFT and RIGHT for move up and down one line. PREV and NEXT could be for move to first line and move to last line. These have the same scopes as the other uses of these buttons.

It's existing functionality of ChoiceBox (change the order of the EPGs and extensions).

[prl]

There are currently no restrictions on where you can navigate in File Commander. That's sort of the point of it.

The default settings are that you enter File Commander with the device selection list as the current list in both panels.

[IanSav]

Hi Adoxa,

I don't think PREV or NEXT are appropriate selections for moving an item up or down one entry. I think LEFT and RIGHT for move up and down one line. PREV and NEXT could be for move to first line and move to last line. These have the same scopes as the other uses of these buttons.

It's existing functionality of ChoiceBox (change the order of the EPGs and extensions).

As I have indicated a number of times all the navigation options will be changing to be uniform across the UI. If you don't use the suggestions now the navigation will change later.

Regards,
Ian.

[peteru]

all the navigation options will be changing

I'll wait until I can see the whole solution and evaluate it. However, it's probably fair to give you a heads up that I do not intend to merge changes that radically overhaul the UI into the 17.5 series firmware. I want the 17.5 series to stay a long term production release and don't want it to undergo a major overhaul now that it is the official release for all machines.

Any major changes will have to wait until the next firmware series. So far I have not started work on 18.x series and don't have any immediate plans to do so.

[IanSav]

Hi PeterU,

That is a shame. OpenPLi and OpenViX are accepting the new UI changes into their development builds now. So far I am getting enthusiastic feedback about the standardisation of the UI and all the new features I have been introducing.

VirtualKeyboard fully embraces the new navigation paradigm. While NumericalTextInput doesn't have any navigation it does fit into the new code structures. All the new Setup developments also now support the new navigation as will all other changes I will be making to Enigma2. This code is going to be fed into OpenPLi and OpenViX. It is disappointing that you are holding Beyonwiz users back and making them wait for all these new enhancements.

Is Jai aware of and behind your decision regarding the future of Beyonwiz firmware updates?

Given your comments and previous treatment of my pull requests I now won't bother submitting any of my code to the Beyonwiz image. I will not waste my time or effort in writing old styled code just to suit you and then wait for you to insist that I fix all that code later when merging becomes a nightmare. I am no longer interested in battling with you to get my code accepted. If you are going to be serious about improving the Beyonwiz image then you can either pull my changes from OpenViX (or OpenPLi if you want to make the appropriate corrections) or I can and will make pull requests directly to the Beyonwiz image if I can be assured that they will be accepted without all the hassles and I will not be wasting my time.

Regards,
Ian.

[prl]

I'd have thought that incorporating these changes at about the same time as OpenPLi and OpenViX incorporated them would make future upstream merges from those two sources easier rather than harder.

[prl]

Here's a patch to try out the new automatic refresh, to see if anything needs changing before I submit it. It also includes the rename fix, as well as enabling 8 for mediainfo, and allows files to be copied/moved to the root (which will be part of the submission). Source included, or check it out here.

I had a look at the code, but I haven't tried it yet. I've been trying to think of a way to make this ability global: to be able to have the same kinds of updates in the media selection screen as well as in the File Commander.

Also, having the updates done via structures that have the lifetime of the Job, rather than the lifetime of the screen, would mean that you'd get the correct updates if you started a file copy job, exited the screen, and then re-entered it. That's a common scenario that causes confusion in the media selection screen.

I've even thought of doing it by having the screen code add callbacks to already-running jobs, possibly by using a callable object with its own callback list as the callback for a Job to get around the fact that a Job can only have a single callback.

I haven't really looked in detail about how to do it, much less try to implement it.

[Paul_oz53]

Hi guys,

It seems the changes to File Commander may still have a little bit of work required, or maybe it's a skin issue.

Rename works fine now on 20180825 Beta but for one detail - long file names are truncated in the display ([Edit: ignore this misread the file target name] and slightly corrupted) although the window seems wide enough for the full name. Changes are not truncated - it is only a display issue.

Rename popup.jpg (39.55 KiB) Viewed 4087 times

Rename popup.jpg (39.55 KiB) Viewed 4087 times

File Commander.jpg (108.73 KiB) Viewed 4087 times

[prl]

It seems the changes to File Commander may still have a little bit of work required, or maybe it's a skin issue.

Rename works fine now on 20180825 Beta but for one detail - long file names are truncated in the display ([Edit: ignore this misread the file target name] and slightly corrupted) although the window seems wide enough for the full name. Changes are not truncated - it is only a display issue.

That behaviour should be unchanged. What "corruption" do you mean?

You can change the display width for the InputBox in FileCommander. It defaults to 40 and can be adjusted to any value between 1 (!?) and 100. In the File Commander main screen, change MENU>Settings>Input length - Filename.

I'm currently working on changes to Components.Input (the actual text input display widget in InputBox and VirtualKeyBoard) that will allow the text displayed to adjust itself to the full width of the available space.

It's not a skin issue.

[IanSav]

Hi Prl,

I look forward to your changes. I think they could improve the UI significantly. (Oh dear a UI change will PeterU allow it? )

Regards,
Ian.

[Paul_oz53]

Thanks Peter, I hadn't explored the settings options so I'll give that a try when I get home.

As for the corruption, l realized I was misreading the source filename. I edited the post to correct that error.

[prl]

As for the corruption, l realized I was misreading the source filename. I edited the post to correct that error.

OK, thanks. I wasn't sure quite which bit of the post the correction referred to.

[adoxa]

I had a look at the code, but I haven't tried it yet. I've been trying to think of a way to make this ability global: to be able to have the same kinds of updates in the media selection screen as well as in the File Commander.

While you think about it, I've submitted my FC approach. I've also submitted a fix to blank out the storage devices header when you move there directly (BACK or 5 with no default).


There's a flaw in the header, in that it shows the current subdirectory (or parent directory), not the directory of the pane. I save /home/root/logs in the right pane, but the header shows /home/root/, as that's what the initially-selected parent directory is. Perhaps if a directory is selected, but the pane isn't, the header should show the current directory (without any stat info)?

[prl]

There's a flaw in the header, in that it shows the current subdirectory (or parent directory), not the directory of the pane. I save /home/root/logs in the right pane, but the header shows /home/root/, as that's what the initially-selected parent directory is. Perhaps if a directory is selected, but the pane isn't, the header should show the current directory (without any stat info)?

The header shows the details for the item in focus in the list, including its full pathname. When that item is the parent directory, for some reason it doesn't show the status details for the parent, but it does show its name.

I'm not sure why it doesn't show status details for the parent. I understand when the "parent" is List of Storage Devices, since that's not actually a directory, just a concoction of enigma2, but a normal parent directory has the same sort of status information as any other directory, right up to the system root, where the parent directory is the root directory (/. and /.. point to the same inode).

I guess you could always make the first entry <This Directory>. That would be consistent with ls -a:

Code: Select all

root@beyonwizu4:~# ls -a
.
..
.bash_history
.cache
.fc-rar-installed
.ssh
enigma2-plugin-systemplugins-FCRenameFix_1_all.ipk
installer.sh
logs
switch_feed_to_beta.sh
tests
uninstaller.sh
root@beyonwizu4:~#

[adoxa]

I'm not sure why it doesn't show status details for the parent.

It can descend, but its index is 0, so pathname is not set, so an exception is raised, which is caught, returning an empty tuple.

I guess you could always make the first entry <This Directory>.

No, I don't like that idea. It still wouldn't help, anyway. E.g. if the current directory is /home/root/ and logs is selected, then the header shows /home/root/logs/. That's fine within the pane, but not so much when you switch to the other pane. Well, that might be a bad example, since you can still infer /home/root. However, if the parent is selected, you'd just get /home/, with no indication of the current directory.

[prl]

I'm not sure why it doesn't show status details for the parent.

It can descend, but its index is 0, so pathname is not set, so an exception is raised, which is caught, returning an empty tuple.

That's not the "why" I was asking about. It was more about why there seems to have been a choice to not display the status of the parent directory.

Also, statInfo() should not return pname in that part of the code. That's a copy/paste error by me. It should probably return an empty list or tuple and whatever it was supposed to achieve should be done in FileCommander.plugin.updateHead().

[adoxa]

Any objections if I change 5 from "Default Folder" to "Location..."?

Submitted.

I've called it "Go to bookmarked folder". The default list (and order) is: the default path (if set); /home/root/; /etc/enigma2/; /usr/share/enigma2/; the default movie path; and "Storage devices" (rather than individual mounts). I've made the menu two lines bigger to fit two new entries: add/remove selected & current folder. It might be better to make the menu smaller, moving the colours and uninstall items to a second page. There was also a bug with the menu, in that the list was wider than the screen, preventing the scroll bar from showing; should this not be accepted a separate fix for that should be made.

[peteru]

I don't like the idea of pre-populating the bookmark list with "system" directories. In particular, /etc/enigma2 or /usr/share/enigma2 should not be in the default list. Power users can easily add those directories if they really need to visit them frequently, but it's not a wise idea to "suggest" these locations to a typical user.

I can see how a user not familiar with the UNIX dir tree conventions could easily mistake the /usr/share path as some kind of special directory used for sharing files and copy recordings there, thus filling up the flash and making the box unusable.

I've removed those dirs from the default list.

[adoxa]

There's a flaw in the header, in that it shows the current subdirectory (or parent directory), not the directory of the pane.

I changed it so the header always displays the current directory. With "Parent Directory" and "List of Storage Devices" there's no info. Selected directories also remove the trailing slash (thus "/path/dir" is "dir" selected in "/path/", whereas "/path/dir/" has its parent selected). I've also titled the storage devices. Any objections?

This is the new code for each updateHead:

Code: Select all

				filename = self[side].getFilename()
				if self[side].canDescent():
					filename = filename and filename[len(dir):-1] or ""
				self[side + "_head1"].text = dir + filename
				self[side + "_head2"].updateList(self.statInfo(self[side]))
			else:
				self[side + "_head1"].text = _("Storage devices")