FileSieve 4

For example, the following path:

\\server\path\

The first slash of the two leading slashes is being treated as a switch character and consequently is being stripped from the UNC, so textForge cannot open it correctly.

There is now a check within the BiQubic.Suite.Core.AppFramework library that now determines if a command-line parameter starts with a double-slash; if it does, then it is treated as an unswitched parameter.

The default value of the MaximumFilesize property was incorrectly set to 0 which resulted in the property’s value being set back to 0 if it was ever changed to -1 (minus one meaning that no max size check should be performed).

There will be additional settings available that apply to the overwrite rules, specifically how FileSieve should determine what constitutes an ‘identical’ file. For example, compare just file names; or file names, date stamps; or hashing; or all of those.

The Destination area of FileSieve simply isn’t big enough to contain these extra options - and it would look like a mess even if it was - so a new window will be added to contain them.

When FileSieve (FS) encounters a file that already exists in the destination and the user has set some overwrite rules, FS generates a filename based on the entered Rename Mask and tests to see if that exists. If it does, it then generates another name (based on the # code (%file_number% in 4.36); that hash symbol represents a number that is incremented each time an on-disk collision is detected) and tests that.

Due to a bug in 4.35 and earlier (unknown at which point the bug was introduced but it may have been due to some major changes to the engine code - specifically the Prescanning step - so FS can gather additional metadata on files to give plugins more control and flexibility), the first attempt at generating a filename was always successful regardless of if that file existed on-disk.

The reason for this is that it was using the wrong path to check for the file’s existence.

Fixed in 4.36.

Add additional overwrite codes to the Overwrite Rules Editor, such as %created_year% (original file’s created date year), %created_month%, and so on.

When the Overwrite Rules Editor was originally implemented, FileSieve didn’t collect comprehensive metadata on the files it processed, such as file dates and the like. It’s been a while since metadata collection was added, so the rules editor needs to be updated to support that information.

IMPORTANT CHANGES TO SOME EXISTING CODES

 %file% is now %file_name%
 %ext% is now %file_ext%
 # is now %file_number%

When files are copied or moved to their destination, their date stamps (Created, Modified, Written) are changed to the date and time of when FileSieve processed them.

This may sound like a bug (and initially did to me when this was reported), but the files are that copied/moved are entirely new files - not the originals that are being processed. FileSieve uses its own I/O code that supports buffer optimisation, async I/O, and operation progress; so it isn’t manually setting the date/time stamps like the OS does when it itself performs the copy or move.

This task entry is to add a new feature to FileSieve that preserves those date and time stamps.

I’m somewhat undecided whether this should be a core FileSieve setting or implemented as a Modifier. FileSieve settings aren’t configurable on a per-Profile basis and so will more than likely be a Modifier. The only potential issue with it being a Modifier is that FileSieve doesn’t set “default” Modifiers; coupling a FileSieve setting with the Modifier itself will provide user convenience in-spite of that fact FileSieve will become (loosely) coupled with a plugin.

Thanks to Olivier C. for reporting

Message: Object reference not set to an instance of an object.

Runtime: 2485 seconds

at FS.Main.Core.Process.set_IsPaused(Boolean value)
at FS.Main.Core.Process.Stop()
at FS.Main.Forms.Main.cmdProcessing_Stop_Click(Object sender, EventArgs e)
at System.Windows.Forms.Control.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
at System.Windows.Forms.Control.WmMouseUp(Message m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message m)
at System.Windows.Forms.ButtonBase.WndProc(Message m)
at System.Windows.Forms.Button.WndProc(Message m)
at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message m)
at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)

Submitted via the Pique exception service; thanks to the submitter!

Due to moving over to the new, enhanced tooltips in 4.29, the banner in the main window isn’t having its default “This does something or whatever.” text updated to reflect the selected tab in the menu.

After a MetroForm has been opened, all ToolStrip renderers are being drawn with the system’s accent colour.

Right-clicking on an item (or set of items) within the Simulation window and selecting Exclude this from further processing should add that item(s) as a disallowed rule within that item’s Source Item.

This will require adding SourceItem to FS.Plugins.Item so the item’s parent can be resolved.

With the Classification method and the Move copy mode selected, FileSieve errors while trying to move files to the destination. This shouldn’t(!) be specific to just the Classification method as it itself doesn’t actually do any I/O of its own, so any method could expose this problem.

The Process.ProcessItem() log entry also doesn’t actually say what the reason for the failure was. Presumably this is because the reason isn’t easily accessible downstream. Either way, get that fixed as well.

Thanks to M. Buckton for reporting

Running FileSieve with the -portable parameter causes it to crash; this is likely down to the new automation support, especially as there’s a delay for a second or two before the crash happens as that’s when Automation is invoked.

Thanks to K. Armstrong for reporting

Now that FileSieve supports automation, being able to modify settings could prove useful.

This would primarily be used from within an AutoFile, but as it works the same way as command-line automation, it would be accessible from both.

Some kind of generic way of setting a value would be best. For example (temporary names and syntax):

  set:key=value

This would set the key setting to value.

Would the Profile command be changed to use this? Or is that an actual command, whereas the above is just a property change? Not sure.

Now that command-line automation has been added ( FS#11 ), it should be easy(ish!) to allow the automation via other means such as a text file.

Since .NET 4.6.2, there is support for long path and file names (over ~230 chars).

Add support for determining if support is enabled at the OS-level (gpedit.msc - Local Computer Policy - Computer Configuration - Administrative Templates - All Settings - Enable Win32 long paths; see this post here for further information) and taking the appropriate action if it isn’t, such not processing items that are over MAX_PATH.

Add information to the documentation for enabling the long paths property via GPEdit.

Support for getting files that have a minimum and a maximum file size has been added to the IO.Directory namespace; expose this property within the Source Item Editor.

Add a feature to allow the “cleaning” of any used Method/Modifier data from Profiles.

Selecting a Method will cause FileSieve to insert its settings (either changed by the user or not) within the current Profile. This causes FileSieve to remember any changes that have been made to any Methods.

The downside of this is that the Profile will increase in size with potentially unused data.

This description makes it sound like Profiles are bloating to megabytes in size; this is not the case. Profiles that are ordinarily a couple of kilobytes in size can increase to approximately 50KB in size, with most of this coming from the Classification plugin.

50KB is absolutely nothing, admittedly, but still.

There’s something very wrong with the whole Profile and Method/Modifier interaction in regards to the currently active settings.

Profile.Active.ActiveMethod.MethodSettings contains the correct settings
Profile.Active.ActiveMethod.Method.Settings contains the INCORRECT settings

When there are multiple profiles defined, it appears the Method settings for one (maybe the first) profile is loaded despite another profile having been auto-selected at during startup.

Removing the extra profile results in the Method settings being correctly loaded for the profile.

Further testing is required to see if Modifiers - and anything else - are affected.

EDIT: Can’t seem to delete this task, so instead see the replacement task  FS#281 .

Reported by Douglas Wofford in Task#273:

“The second issue is the result of a quirk with how Windows itself corrects an invalid character. There are few album names in my files that end with a period [.], for example R.E.V.O. or Shakira. Windows allows you to type in (or submit) a path like this ‘F:\Music\PlaylistReorg\RockPop\Walk Off The Earth\R.E.V.O. \’ with no error. The path that gets created, however, ‘F:\Music\PlaylistReorg\RockPop\Walk Off The Earth\R.E.V.O\’ – Windows strips off the final . and space. As result, when FileSieve attempts to move or copy the file into the directory Windows confirmed was created, the directory doesn’t exist and the move/copy fails. This is a tough one since it’s kind of weird one-off problem.”

Reported by Douglas Wofford in Task#273:

“The issue is with the option to have a slash character in Genres replaced with a -. This is not new to the new build – it was happening before – but I had missed it. I’ve included a log file showing what happens. Basically, the routine is inserted the - character some of the time, but most of time it’s just eliminating the character. So, you will see some path names as F:\Music\Rock-Pop\... but most path names as F:\Music\RockPop\... or Blues-Folk and BluesFolk. Unfortunately, I don’t see any kind of pattern related to how the Genre name is handled.”

This issue is related to (part of?) existing issue 269.

The following combinations lead to unexpected results in the location of the created directories. I am using modifiers to replace characters that are invalid for Windows directory names. They are working as expected except for the quirks identified below.

Step 2 - Destination: Set to F:\
Step 3 - Options - Method - Metadata - Tag:

 Name Mask:  \Music\PlaylistReorg\%GENRES%\%ALBUM_ARTIST_FIRST%\%ALBUM_TITLE%\
 Checked: Replace any slash characters encountered within the Genres tag.

Step 3 - Options - Modifiers:

 [All modifier rules created to handle characters in ALBUM_ARTIST_FIRST and ALBUM_TITLE that are invalid for Windows directory names]
 Find : Replace With -
 Find ? Replace With NULL
 Find # Replace With NULL
 Find & Replace With and
 Find , Replace With NULL
 Find + Replace With and
 Find * Replace With -

Renaming Results:

 New file name is set to \Music\PlaylistReorg\... and saved to the root of the C: drive

If the name mask is changed to F:\Music\PlaylistReorg\%GENRES%\%ALBUM_ARTIST_FIRST%\%ALBUM_TITLE%\

 [Explicitly using the Drive letter]

Renaming Results:

 New file name is set to F:\F-\Music\PlaylistReorg\... and saved to the correct drive (F:) but underneath a new directory F-

Finally, if the name mask is changed to Music\PlaylistReorg\%GENRES%\%ALBUM_ARTIST_FIRST%\%ALBUM_TITLE%\ [removing the leading \]

Renaming Results:

 New file name is set to F:\F-\Music\PlaylistReorg\... and saved to the correct drive (F:) but underneath a new directory F-

The modifier rules work correctly on the %ALBUM_ARTIST_FIRST% and %ALBUM_TITLE% variables to create the correct sub-directories under F-. The check box option correctly corrects the / character in the %GENRES% value to the - character as well.

The issue appears to be restricted to how the drive letter is being handled. The outcome of all of the combinations above results in the correct reorganization of the files from a single flat directory into multiple correct sub-directories. The new sub-directories end up, however, either in the root directory of the default C: drive or in a new directory F- on the F: drive.

Add a Mirror Method that simply mirrors the structure of each source item to the destination.

A straight-up file copy, in simplistic terms. Obviously won’t be as simple to code, but that’s it in a nutshell.

Not a walnut, because they taste weird.

When processing, do something with files that have illegal characters in their filename.

Despite Windows not allowing such chars, they can still be created via some programs. FileSieve should have a setting (plugin?) that can dictate what should happen to them - probably replace them with valid alternatives.

Known places that throw (caught) exceptions:

• FileSieve4.Plugin\Item.UpdatePaths() : line 262
• FileSieve4.Main\Core\Process.ProcessItem() : line 751 (CreateDirectory)

While FileSieve is processing files, disable the About... menu item within the main window’s sysmenu. If it’s too much faffing about, simply ignore the click instead.

The Verify link within the Source Item Editor needs to perform validation checks on the regular expression Rules to ensure they are correct; if they aren’t, notify the user.

When adding/removing/clearing nodes within the tvwFolders TreeView, the Parent property isn’t being correctly set which results in an error when using drag & drop to move those Folders around.

For now, FileSieve 4.15 just displays a friendly error message and notes that the bug will be fixed within the next version.

Add more command-line parameters.

Exception: Value of ‘720’ is not valid for ‘Value’. ‘Value’ should be between ‘minimum’ and ‘maximum’. Parameter name: Value

at System.Windows.Forms.ProgressBar.set_Value(Int32 value)
at FS.Main.Core.Results.ResultsControls.UpdateProgressBar()

Add documentation for Triggers.

Using the Profile Manager, clicking on New but keeping the default New Profile name and hitting enter doesn’t rename the Guid-based on-disk file to “New Profile” and instead keeps the generated Guid.

Discovered while implementing the new Trigger system.