RadioDJ - Free Radio Automation Software Forum

RadioDJ v1.7+ => v1.7+ - Support Forum => Topic started by: spirosk on November 18, 2014, 04:42:35 PM

Title: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: spirosk on November 18, 2014, 04:42:35 PM
When importing folder why i get "[ALREADY EXISTS]....

Tracks are the same but:

1.In diffent folders
2.Applyed  different Genre
3.Applyed different Subcategory

Is this normal ????
Thank's
Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: Marius on November 18, 2014, 06:08:07 PM
The same file cannot be imported more than once, because you would get repeated plays for the same track.
Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: spirosk on November 18, 2014, 07:42:38 PM
Since the tracks included in different folders (path)  with different Genres & Subcategories
should managed  as different  tracks. Repaets should controled from an option
(looked for one but did find)  "Don't repeat same track for the next [hh:mm:]"
Jazler  behaves like this and it's absolutely right… 

Is there any way to solve this anyway and make it behaves like a different track ?
Thank you very much ….
Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: Marius on November 18, 2014, 08:33:59 PM
As long is the same filename, no.
Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: spirosk on November 19, 2014, 07:51:06 AM
So ...
I have the rotation:

-Jinle
-Indie/Pop
-IndieSoft
-IndieRock
......
......

and some tracks belong all tree Genres .

So these tracks must play form -Indie/Pop or from -IndieSoft or from -IndieRock but
since tracks can stored only once this can't happend....
You see what i'am saying ?????
That's my problem .....



Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: AndyDeGroo on November 21, 2014, 01:28:22 AM
I think that you should use genres for genres, not categories as genres. And you should know that Marius is working hard to add the so much requested feature of multiple categories for each songs. Believe me, it is not an easy few-lines-of-code fix and it won't be ready tomorrow.

Let me explain how RadioDJ categories, sub categories work: First of all, they are not folders. Second, songs are not copied into database or moved anywhere. Files stay where they were and that's why Marius referred to the file path, which must be unique among imported files. If you really, really, really need to import file multiple time and don't want to duplicate them, you should use file system junctions or so called symlinks, to make the file paths look unique. Easiest way to create symlinks or junctions is by using Link Shell Extension.
I hope it helps but feel free to ask if anything is not clear.
Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: spirosk on November 21, 2014, 09:14:22 AM
Hi AndyDeGroo !
Thank you very must for your reply ....
I know very very well how hard some times is just 'simple things' is programming to happen since
I’ve been a programmer too so I feel  must respect for any programmer  ….

Many songs categorized in  more than one Category or Genre so if you categorize
one track in only one Genre it will play again when rotation of the specific Genre will come around .

But if the same track categorized in more Genres, somehow, will play more often and in more Genres witch is normal….
I believe this can happened only if tracks in different path managed as different.
Thank you !
:)
Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: gstark on November 28, 2014, 06:01:56 AM
The same file cannot be imported more than once, because you would get repeated plays for the same track.

I'm seeing a similar issue to that described by the original poster.

I'm importng a large library - > 24000 tracks - and when importing these files for the very first time I am seeing this message on several of the files.

It may be that the file names are the same, but these files are located in different folders, and there is no way that these are actually the same physical file in the filesystem.

And being the first time import, there is simply no way that the actual physical file - using the full file-path file name - will have already been imported into the library.

So, there seems to maybe be some sort of issue with the imprt process, in that it sometimes gives a false positive for what it's believing to be a duplicate file.
Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: palinst on December 02, 2014, 07:06:14 PM
As long is the same filename, no.

Just to be clear, we're talking the same "filename" or the full "pathname". So, for example, if you
had the following:

C:\MEDIA\SONGS\CHRISTMAS\ARTIST1\JOYTOWORLD.MP3
C:\MEDIA\SONGS\CHRISTMAS\ARTIST2\JOYTOWORLD.MP3

Would both of these import, or would the "already exists" message appear for Artist 2's Joy to the World?
Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: Marius on December 02, 2014, 08:12:50 PM
No matter what path is, if the filename already exists it will not be imported again.
Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: palinst on December 02, 2014, 08:25:26 PM
No matter what path is, if the filename already exists it will not be imported again.

That's a tough nut with Christmas songs, since there are 100 versions of the same 25 songs. But that explains the problem we're having.

Thank you for clearing that up.
Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: gstark on December 03, 2014, 01:12:34 AM
That's a tough nut with Christmas songs, since there are 100 versions of the same 25 songs. But that explains the problem we're having.

It's also pretty bad with standards.

How many different versions of "Sunny Side Of The Street" are there?

"Take Five" ?

"All Blues" ?

"Eleanor Rigby" ?

Just to name a few. Using just the file name is, with all due respect, not the best way to do this, as it presents a very clear imposition on just what a person may be able to import.

Putting the track number into the song title filename is a very poor workaround, as the song title does not include ... any track number.

Using the full path would be the correct, and ideal, solution, I believe, as it makes every instance of a file a unique occurrence, and it's more representative of the real world situation that we all encounter.

My music collection consists of around 24500 titles, and I'm just starting to dupilcate a lot of that in HD, so, again, the titles will be the same, although the paths will be quite different.

I can already see that there's about 1000 tracks that have been not imported because they've been, incorrectly, IMHO, flagged as dupes, and from what I'm seeing, any new HD tracks that I try to bring in will also be rejected.

Again, and with the greatest of respect, I believe that this does need to be reviewed.
Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: fuzzyfaraway on December 03, 2014, 03:59:41 AM
I rename all files to <artist> - <track-name> before importing to the database. It greatly simplifies things. I have around 700 Christmas tracks with many duplicate song names, but because of the file name format, they all imported without trouble. I use the "Bulk Rename Utility" to change a whole album at once.

Even some commercially available music libraries come with this file name format, since several automation systems require it.
 
John T
Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: gstark on December 03, 2014, 06:55:18 AM
But why would you have the artist name in the filename, when you would also have the artist name in the folder structure?

/Artist Name/Album Name/Song Title

seems to me to be the best and most logical way of addressing the filesystem's structure. It simplifies things, and permits you to group most of an artist's work together.

I like to keep things simple and clean, and it'll take a lot to convince me that the nine different versions of "A Foggy Day" (to pick just one work), each of them by different artists, with different lengths, instrumentation and arrangements, are actually duplicates of one another. :)
Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: djginod on December 03, 2014, 05:49:12 PM
Yeah, but how many songs, do not have the title, Love, Lovesong, I Love you etc.... your way of working is very organised, but not the best method I guess for this program. All my songs are in this format: Artistname - Title

MP3tag can help you with this foe example.
Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: gstark on December 05, 2014, 02:28:37 AM
your way of working is very organised, but not the best method I guess for this program.

Yes, what your saying is correct, and therein lies the problem: the software should be designed to help us, as the user.

Not the other way around. I'm actually a professional software developer with over 30 years' experience, and it's exactly this sort of issue that I'm frequently called in to address.

For what it's worh, MediaMonkey tell me that I have 27 different versions of Gershwyn's "Summertime", and SAM also tells me that I have 27 different versions of the same song. RadioDJ reports just 15. This means that almost half of the versions of this track that I have, including the great Ella Fitzgerald version, are not available to me within RadioDJ.

Is this a good thing? I don't believe so.

The core issue here is not one of tags, but of file management, and there's a few different ways it could easily be addressed. Very easily.

1: Check for dupes on the basis of the full pathname of the file. By definition, there can not be any duplicates.

Alernatively,

2: Store a MD5 hash of the actual file in the database. By calculating a new MD5 hash on any file as it's being imported, and comparing the hash value with that stored. At any time, if the hash value is the same, then the file will be a duplicate.

Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: AndyDeGroo on December 15, 2014, 06:47:53 AM
1: Check for dupes on the basis of the full pathname of the file. By definition, there can not be any duplicates.
RadioDJ actually is using FULL PATH to decide if track is a duplicate. There is a unique index on `path` column in `songs` table.
It's somewhat strange for an experienced developer not inspect database to understand how it all works.
Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: gstark on December 16, 2014, 04:04:53 AM
RadioDJ actually is using FULL PATH to decide if track is a duplicate. There is a unique index on `path` column in `songs` table.
It's somewhat strange for an experienced developer not inspect database to understand how it all works.

Just because there's an index in the database does not mean that it's used for this purpose.  I don't have access to the source code, so I'm not able to verify what is actually happening,  and I would never presume to make the leap that joins the dots as you're suggesting as I simply have no evidence to make such an assertion. I

I'm going on the basis of my own observations,  where non duplicate files (based upon the full file path) have been flagged as duplicates, plus the comments of other people,  including posts by Marius within this thread,  which state that only the file name is used for this check. 

I would respectfully suggest that if Marius says this, then that would be a rather authoritative statement. :)

Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: Marius on December 16, 2014, 09:30:07 AM
Path column is set to unique if this helps.
Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: gstark on December 17, 2014, 01:03:16 AM
Ok, Marius,

Unique is good, and it's probably how I'd expect it to be, but now I'm a little bit confused. :)

Path column is set to unique if this helps.

Now, for the purposes of this thread, your earlier comment ...

No matter what path is, if the filename already exists it will not be imported again.

says (to me) that the path is not used to determine the uniqueness of any file, and that the uniqueness of any file is governed solely by the filename ...

But yet, and as pointed out by AndyDeGroo, you have a unique index in the db, which could easily, and more correctly, determine the true uniqueness of any file.

Your earlier comment suggests that this index is not used for this purpose, (I'd expect it's used to find the file for playing purposes, amongst other things) and it's not for me to query your reasons, but it seems that, if there was a way that this index could also be used for this purpose, rather than just the filename, then perhaps there might be a better outcome here for all concerned? 

Thanks for your help.

Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: AndyDeGroo on December 17, 2014, 04:55:51 AM
As a RadioDJ user with one year experience and developer of one plugin which imports files, I can assure you that full path is used for checking if file is a duplicate.

I can see the source code and you could as well, because it is way too easy to disassemble .Net assemblies.
What RadioDJ does in Tracks.ImportTrack method is this:
Code: [Select]
if (trackData.ID == 0)
{
text = "INSERT IGNORE INTO `songs` SET ";
}
else
{
text = "UPDATE `songs` SET ";
}
The text variable is a string used for SQL query which gets concatenated to form full query for inserting or updating track info.

I hope that it's clear now and we can put this discussion to rest.
Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: Marius on December 17, 2014, 08:02:48 AM
Beside that, in the import code there is a part which it checks also for the filename in the path column. If it's found one, the track will not be imported.
You must understand that this is required because many times the same track is added to more than one album/compilation and if i allow multiple import only based by the path, the tracks will be treated as different and you will end up playing the same track more often than you set in track repeat interval.
Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: AndyDeGroo on December 17, 2014, 09:01:29 AM
Oh, I spoke too soon. Now I see that rdjInterface.Tracks.TrackExists(string filePath) method is actually checking file name at the end of `path` field:
Code: [Select]
string sql = "SELECT `path` FROM `songs` WHERE `path` LIKE '%" + rdjInterface.Utils.QueryFix(Path.GetFileName(filePath)) + "' LIMIT 1;";
Similarly rdjInterface.Tracks.TrackExists(string artist, string title, string rrr) is matching `artist` and `title` fields. Whatever the third argument rrr is.
Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: gstark on December 19, 2014, 01:30:47 AM
Beside that, in the import code there is a part which it checks also for the filename in the path column. If it's found one, the track will not be imported.
You must understand that this is required because many times the same track is added to more than one album/compilation and if i allow multiple import only based by the path, the tracks will be treated as different and you will end up playing the same track more often than you set in track repeat interval.

Hi Marius,

Thanks for that clarification.

That solution works for those who use the program in the manner that you've suggested.

But I don't think that all of your users do it in this manner.

As I've said in an earlier post, I will have a large number of tracks, all with the same trackname, but they will all be different tracks. Different artists, for instance.

Or different performances.

Or perhaps the same album, but the original album, and a second (or third) version of the same album, perhaps remastered, or maybe a HD version of it.

In each of the instances that I've outlined here, the tracks may correctly have the same trackname, but they will mostly be different tracks, and they all should, more correctly, be imported is such.

As it stands at the moment, I have (for example) Summertime by Ella Fitzgerald, Summertime by Big Joe Turner, Summertime by Janis Joplin ... Herbie Hancock, Toots Thielemans, Bill Evans, Brian Wilson, George Benson, Billie Holiday, Sarah Vaughan  ... 27 different versions of Summertime in fact, but while they are all versions of the same song, written by George Gershwyn,  they are each very different tracks. Identifying each of these only by the trackname property of the filepath fails to take into account the fact that each of these are very different performances, by, as you can see, very different artists.

Perhaps you could make this part of the import process subject to a setting in the options, so that we can choose whether we wish to define the term "duplicate tracks" by purely the filename, or not?

And yes, I'm perfectly happy to have actual duplicates of some tracks, such as from compilations, in my DB.

Cheers.

Title: Re: Why [ALREADY EXISTS] when folder is different Genre is different ????
Post by: power_fm on December 21, 2014, 08:38:07 PM
This might sound like double-work, but I have created a single folder on an external drive that has all the songs we use. It only has the song title as the file name. When we want to add new songs to RadioDJ we put the songs into this folder first to see if Windows detects the same file name as we copy across. I take note of any file name clashes and use mp3tag to change them. All this happens before we load the songs into Radio DJ.

We call it the "superlist" and the bonus is that we have a backup of all our music in a separate location to the Radio DJ library.