Jump to content

Incorrectly restoring short names


pls

Recommended Posts

I apolgize for the length of this, but I promise it will be worth your while to read it.

 

 

 

I recently (a couple of hours abo) posted this message to the Retrospect 5.6 forum:

 

 

 

-------------------------------------------------------

 

Environment:

 

Retrospect 5.6 Windows

 

Windows 2000 Professional

 

Intel Celeron 800

 

512 MB memory

 

IBM 75 GB hard drive

 

 

 

 

 

The second (not operating system) drive was going flaky. So I replaced the drive and did a restore drive using Retrospect.

 

 

 

When the restore completed I discovered that almost none of the applications on the drive worked.

 

 

 

I'm a software developer, so I was able to figure out why: Retrospect didn't restore short file names, so any ActiveX control registered using a short name was not found.

 

 

 

To be specific, the short name for the Microsoft Money executable before the restore was

 

G:\PROGRA~1\MICROS~2\SYSTEM\MSMONEY.EXE

 

after restore, the path was

 

G:\PROGRA~1\MICROS~3\SYSTEM\MSMONEY.EXE

 

 

 

It seems clear to me that Retrospect didn't save and restore the short name, but let it be generated when the file was restored. Since the files are restored in a different order from their original creation, the sort names are assigned differently.

 

 

 

So I now have a machine where nothing works. And a very limited set of choices because I don't have a useful backup like I though I did. This is a serious enough error to make Retrospect useless on the Windows OS.

 

 

 

I can't believe I'm the first to spot this. Do I have a setting wrong? Is there a workaround? Or did Dantz release a backup program that can't restore?

 

 

 

Thanks,

 

++PLS

 

 

 

P.S. This also tells me that Retroepect isn't using the BackupRead and BackupWrite api, which is the only correct way to backup and restore on an NTFS file system. In other words, there are other things that Retrospect will fail to restore correctly.

 

 

 

---------------------------------

 

 

 

Here is some additional information.

 

 

 

First, short names explained...

 

 

 

In MSDOS and 16 bit Windows file names were limited to the infamous 8.3 characters. That is, a file name of up to 8 characters and an extension up to 3 characters and nothing else. This name format is also written into most 16 bit apps. When 32 bit Windows and long file names appeared, Microsoft didn't replace the short 8.3 name, but kept it for backwards compatability and added the long name as an additional name. So all the way up to Windows XP and NTFS, all files have two names, a short and a long.

 

 

 

If you create a file and path that fits within the 8.3 specs, you get only a short name. If you create a file and path that doesn't fit, the long name will be what you specified and the system will generate the short name. You can see the short names by opening a command line window and typing "dir /x".

 

 

 

Sadly the short name didn't say just for compatability.

 

 

 

Any ActiveX component has to be registered in the system registry. This regisration includes the path of the executable (or dll or ocx) file containing the component. Some installers will register the component using the short file name. If the short name changes, the system won't be able to find the component and any application using it will fail. Short names are important. And any backup program mst back up the short name and restore the file with the same long AND short names.

 

 

 

I looked through this forum and the 5.6 forum to see if this problem had beem reported. I didn't find it. I did find a significant number of questions along the line of "I restored these files and this application didn't work". I've just described why.

 

 

 

How to test...

 

 

 

You can demonstrate this problem by doing the following:

 

 

 

1. Create the directory \Temp\RetroTest on an NTFS volume.

 

2. Inside that, create the directory "Application Directory C".

 

3. In the same place, create the directory "Application Directory B".

 

4. Likewise, create "Application Directory A". The order is important.

 

5. Open a command line window and navigate to \Temp\RetroTest

 

6. Type "dir /x". You will see something line this:

 

Short Name Long Name

 

APPLIC~3 Application Directory A

 

APPLIC~2 Application Directory B

 

APPLIC~1 Application Directory C

 

 

 

7. Close your command line window and make sure explorer is not displaying any of these directories.

 

8. Open Retrospect.

 

9. Create a new backup set.

 

10. Create a subvolume for \Temp\RetroTest.

 

11. Backup the subvolume to the new backup set.

 

12. Open a Windows Explorer window, navigate to \temp\RetoRest, and delete the three subdirectories.

 

13. Open Retrospect and do a Restore Disk on the subvolume.

 

14. Notice that the three directories reappear.

 

15. Open a command line window, navigate to \temp\RetroText, and type "dir /x". You will see something like...

 

Short Name Long Name

 

APPLIC~1 Application Directory A

 

APPLIC~2 Application Directory B

 

APPLIC~3 Application Directory C

 

 

 

Notice the short names are different. Any ActiveX components in Applications A and C now cannot be used, shortcuts that use the short names won't work, other applications that store the short name won't work, and so on.

 

 

 

I would very much like someone who has retrospect 6 try this and see if it has the same problem. I suspect it does.

 

 

 

Thanks for your attention,

 

Paul Schauble

 

 

Link to comment
Share on other sites

  • 3 weeks later...
  • 2 weeks later...

Wow.

 

 

 

Glad I read this before installing RS on any systems belonging to clients. RS is under consideration by 3 clients right now. I've been concerned about another issue, but this really trumps any problem I've had. Tomorrow I'll advise these clients to either wait for a fix or use Backup Exec--software I really don't like half as much as RS, but have used successfully for years. Evidently Dantz's own testing hasn't been thorough enough to find this problem, which must be fairly common given all the MICROS~*'s on a typical PC.

 

 

 

Scary.

 

 

 

Quote:

Legendary Reliability - We call it backup software, but Retrospect Express is really restore software. Our philosophy that restore matters above all else means that you can count on Retrospect Express when you least expect it, because that's when you'll need it the most.

 


 

 

 

Ironic, isn't it.

 

 

 

And no replies from Dantz since 12/30/2002? I'll be watching this thread even more closely than my own.

 

 

 

Jeff

Link to comment
Share on other sites

At this time Retrospect does not store short names or do anything to set them. This will be changing in a future release. The reason that this normally isn't a problem is that short names in the system tend to be unique, and the OS is very consistent about assigning them. So we tend to preserve short names even though we don't do it by design. Retrospect has restored 1000's of PC's successfully without a problem.

 

Here's one example of how to get Retrospect to permute the short names in a file system:

 

1. In an empty folder, create the following files in this order:

My file with a long name

My file with a lazy name

My file with a loaded name

 

2. Examine the files' short names. In the same order as above, they're:

myfile~1

myfile~2

myfile~3

 

3. Back up the folder as a subvolume. Retrospect backs them up

alphabetically, in this order:

My file with a lazy name (myfile~2)

My file with a long name (myfile~3)

My file with a loaded name (myfile~1)

 

4. Now restore the files to an empty subvolume. The files get restored in alphabetically, and the system assigns short names:

My file with a lazy name (myfile~1)

My file with a long name (myfile~2)

My file with a loaded name (myfile~3)

 

Note that all of the files now have different short names. Note also that the same thing happens if you just copy & paste the subvolume in Windows Explorer.

 

Dantz recognizes that this can be an issue for some users, and we are working on a solution.

 

Link to comment
Share on other sites

Mayoff, thanks for your reply.

 

Quote:

short names in the system tend to be unique, and the OS is very consistent about assigning them

 


 

Not good enough. My take on that statement is that if Dantz considered the issue at all in designing RS, they were satisfied with an untested (and wrong) assumption.

 

Quote:

Note that all of the files now have different short names. Note also that the same thing happens if you just copy & paste the subvolume in Windows Explorer.

 


 

It will also happen from a command prompt. But Windows Explorer is not generally recommended for backup & restore.

 

I hope the fix is in a near future release, because my own system uses RS6 and I have confirmed through registry examination that I would not be able to restore correctly.

 

Like the original poster, I sell my technical expertise for a living, and know from long experience with 32-bit Windows that this problem makes it a crapshoot whether RS can restore a Windows computer to full operation.

 

Unless one installs software in alphabetical order, *something* is almost certainly going to break. 16-bit software is even more vulnerable because all they ever see is SFNs.

 

Some users may never discover the missing functionality and consider the restore successful. Others may not discover the problem until long after the restore and not associate the failure with Retrospect.

 

I admit there will be some that will be lucky and have a 100% successful restore. No statistics available to "back this up" so to speak, but my gut tells me these people will be in the minority.

 

This is a serious problem. To me, Dantz's credibility rests on how they respond to this bug. Your reply concerns me because parts of it downplay the risk. I think this is the kind of thing that merits at least a prominent link to an FAQ on the subject on your front page which fully discloses the risk, and a free patch (or free upgrade) available to all users of all versions of all affected products. This remedy should be Dantz's top priority.

 

Jeff

Link to comment
Share on other sites

Quote:

Here's one example of how to get Retrospect to permute the short names in a file system:

 

1. In an empty folder, create the following files in this order:

My file with a long name

My file with a lazy name

My file with a loaded name

 

 


 

Here is another way to get the same trouble (I imagine): this example is just to illustrate that nothing particularly unusual needs to happen.

 

1. Buy a new PC with Windows installed

2. Install Microsoft Office

3. Install Microsoft Encarta

4. Install Microsoft Whatever

 

Your "C:\Program Files" will now contain quite a few directories whose short names look like MICROS~1, MICROS~2 and so on -- more than three such directories in the above example. This is the situation of almost all of your Windows customers, I would have guessed. My registry appears to contain a vast number of references to MICROS~3, the short name of Microsoft Office on my machine. What will happen if my hard drive dies?

Link to comment
Share on other sites

  • 3 weeks later...
  • 2 weeks later...

What does "next release" mean? Version 7.0? A patch for 6.0? And a patch for 5.x? I understand the part about not giving an exact release date? How about an inexact release date? I promise not to hold you to the exact date.

 

Are we talking about approximately a week? Approximately a month? Approximately a quarter? Approximately a year?

Link to comment
Share on other sites

"Release" means the next major upgrade to Retrospect.

 

This issue will not be addressed in older versions of Retrospect. As we have said before, 1000's of Windows systems (from home users to corporate users) have been successfully restored without a problem from version 5.0 up to our current 6.0 product.

 

We do understand that some configurations would work better with both long and short names backed up, and we are adding that functionality in our next major upgrade to Retrospect. I can not provide you with a timeframe for when that version will be available.

 

 

Link to comment
Share on other sites

The next "major upgrade"????? You mean we're going to PAY for this bug fix????????

 

 

 

As I have asked before, how many more 1000s have not been restored correctly? Dantz is crazy if they think they can rationalize this as a trivial problem. Much smaller flaws are routinely addressed by other (better) backup vendors in patches released approximately quarterly.

 

 

 

Your "snapshot" catalog is brilliant, but if the underlying backup data is worthless, so is the catalog. To turn a phrase, "Garbage backed up, garbage restored." Dantz is playing Russian roulette with my data. Accordingly, Dantz has lost all credibility with me as a data security vendor.

Link to comment
Share on other sites

  • 2 weeks later...

All Windows operating systems since Windows 95 support long file names. While the operating system has long supported long file names, certain applications and components require short names in rare circumstances.

 

While Retrospect uses the Microsoft provided Backup APIs (BackupRead and BackupWrite) to back up data on all Windows NT based operating systems (Windows NT, 2000, XP, and Windows Server 2003), these Microsoft Backup APIs do not backup or restore short names.

 

Microsoft did not have an API to set or restore short names until they released Windows XP. The new API is SetFileShortName and it works only with NTFS volumes on Windows XP and Windows Server 2003 computers. There is no Microsoft API to restore short names to non-NTFS volumes on Windows XP or at all on older Windows operating systems. This is why customers sometime see different short names for files after they have been restored.

 

The next major revision of Retrospect will add support for tracking short names and restoring them on Windows XP and Windows Server 2003 systems with NTFS volumes. While the new API gives us control over short name restorations, Retrospect will not be able to restore the short name exactly for a file if that short name is already in use by a file. See MSDN for more details on the SetFileShortName API.

 

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/base/setfileshortname.asp

 

We are happy that Microsoft is providing this new API in Windows XP and Windows Server 2003 and look forward to taking advantage of it to solve rare problems where short names are required instead of long file names.

Link to comment
Share on other sites

 

I wonder if the statement "certain applications and components require short names in rare circumstances" reflects the way we would like Windows to be, rather than the way it actually is.

 

My registry contains 202 references to the short file name of Microsoft Office. Most of these look pretty important (or incomprehensible to me the layperson). For some examples:

 

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Extensions]

"xls"="C:\\PROGRA~1\\MICROS~2\\Office10\\EXCEL.EXE"

"doc"="C:\\PROGRA~1\\MICROS~2\\Office10\\WINWORD.EXE ^.doc"

"dot"="C:\\PROGRA~1\\MICROS~2\\Office10\\WINWORD.EXE ^.dot"

"rtf"="C:\\PROGRA~1\\MICROS~2\\Office10\\WINWORD.EXE ^.rtf"

 

Here is an example from another piece of software: Netscape stores its mime types as follows--

 

[HKEY_CURRENT_USER\Software\Netscape\Netscape Navigator\Viewers]

"application/msword"="C:\\PROGRA~1\\MICROS~2\\Office10\\WINWORD.EXE"

 

After a complete restore using Retrospect, this mime type in Netscape will be broken.

 

I'm sorry, but I think phrases like "rare problems where short names are required" are just wishful thinking. For sure, I don't often use Netscape to download and open a word file. Perhaps that's why thorough testing of Retrospect-for-Windows at Dantz did not reveal these problems. But really, you should come clean and say that Retrospect will _not_ restore your machine to its previous state: there will instead be more than 202 small problems in Office alone, most of which you will not encounter very often, but some of which will bite you -- and you may never understand why.

 

Link to comment
Share on other sites

And the reality is, if you are using any operating system other then Windows XP, it is impossible to restore these names (not just with Retrospect), because Microsoft does not provide an API to do so. As we have said before, we are addressing this issue in Windows XP.

 

I hate to say it, but I think we are starting to beat a dead horse with this issue. If you have attempted to restore data and it is failing, lets find out why. Lets identify the problem and see if it is directly related to this short name issue or not. The point of the forum is to help solve problems, lets try and address an issue you are having rather then debate what the Microsoft API's can do and can't do.

Link to comment
Share on other sites

Interesting to watch how the official explanation has evolved over the last 3+ months.

 

I did a couple of quick checks and found:

 

* Win 2000 Server/Backup Exec 9.0/NTFS does not properly restore SFNs.

 

* Win XP/Backup MyPC 4.85 (formerly Backup Exec Desktop)/FAT32 does not properly restore SFNs.

 

Lacking time and resources to run a more comprehensive test, I did not try NTBACKUP, Win 9x BACKUP, or any other vendors' software, but would doubtless see the same thing, so I'll concede this point.

 

I can think of a simple way in which Retrospect and others could provide this functionality without the SetShortFileName API call, even on FAT16/FAT32 partitions, albeit at the cost of a slighly longer restore (VERY worth extra restore time to get it right, though). But I also can't fault Dantz for doing as poorly as everyone else. Hard to believe the issue has not been addressed by now, though. And the responses from Dantz to this issue sure makes it look like Dantz had no idea about this problem either.

 

But I also found that Win XP/Backup MyPC/NTFS does properly restore SFNs, so they know about the new API call and have properly implemented it. Also backs up open files flawlessly at no extra cost using the built-in Microsoft Shadow Volume Copy service that Retrospect launches but never uses. That made my decision easy. Retrospect is gone until (at least) this SFN issue is resolved, and probably until it delivers open file backups at no extra cost.

 

> and look forward to taking advantage of it to solve rare problems where short names are required

 

I still don't think "rare" is the right word, and would like to see 3rd-party verification of this from, independant testing labs.

 

Until then, I'll accept "rare" when I start seeing claims from Dantz to the effect that "Retrospect can restore your computer exactly the way it was during the backup...except in rare cases." But I won't hold my breath on that!

 

It's really a shame...I felt Retrospect was by far the best backup software I'd ever used until this came up. I even had clients ready to switch to it. I'm going to keep an eye on it but will have to approach recommending it a lot more cautiously in the future.

Link to comment
Share on other sites

  • 3 weeks later...

"certain applications and components require short names in rare circumstances."

 

I'm not sure how rare this actually is. A quick look at my system, running WinXP Home Edition, showed me several things that would be affected by this problem on a restore. Here's one. I have the following directories on my system: Here are the long and short names

 

C:\Program Files\Windows Media Player

C:\PROGRA~1\WINDOW~3

 

C:\Program Files\Windows NT

C:\PROGRA~1\WINDOW~1

 

As you can see, even though the long name for the Windows NT directory comes after the media player directory alphabetically, the short name comes first. Retrospect would restore the Media Player directory first, since it restores files alphabetically by long name, so the short filenames would be different. I checked, and there are several registry keys which refer to C:\PROGRA~1\WINDOW~3, that register various ActiveX controls that are part of Media Player, so these would all be broken after a restore. Windows Media Player is hardly a rarely used app, at least by me, so I doubt this would go unnoticed for long.

 

I can't prove this, but I doubt my system is very unusual, and this is just one instance I found in a few minutes. I'm sure there are others; it just depends on what order you happened to install things. For instance, my Program Files directory also contains "Microsoft Office", and "Microsoft frontpage". Again, in this case, since I installed Office before FrontPage, the office directory has short name "MICROS~1, and the FrontPage directory has short name MICROS~2, even though the FrontPage long name comes first. I'm sure Office also has lots of ActiveX controls registered, which would stop working after a restore.

 

I understand your point that MS doesn't provide an API to handle this on pre-XP OSes. I'm just disputing your claim that it's rare for this to be a problem.

 

I can think of a fairly easy trick that should fix this problem even for pre-XP systems. Simply sorting your catalog by short names instead of long names would solve the problem in a lot of cases, since if you restore in the order things appear in the catalog, the short names would at least be generated in the same order as originally. This wouldn't handle cases where there are gaps in the short filename sequence numbers though, as in the above example. Even if you restored the Windows NT directory first, and then the Media Player directory. the Windows NT short name would be WINDOW~1, which is correct, but the Windows Media Player directory would be WINDOW~2, which isn't correct. A second hack would fix this -- during restore, simply create a temporary file after restoring the Windows NT directory, whose short filename would be WINDOW~2, then restore the Windows Media Player Directory, which will now get short name WINDOW~3, and then delete the temp file.

 

If you use the above trick in your next version, can I get a free upgrade? smile.gif

 

Adam

Link to comment
Share on other sites

My take, FWIW: I wouldn't consider using a backup program to restore installed executables or DLL's. The registry is complicated and just stuffing into place registered files is likely to be no end of problems, as some users have discovered.

 

I use Restrospect to back up data files - client data, my personal files, photos, that sort of thing - that cannot otherwise easily replaced. OK, and the registry too, in case I need it. For programs, it simply MUST be more reliable to reinstall, easy enough if you have the original CD (and you should). If I had a complete disk crash, I would reformat and reinstall everything the right way, then use Retrospect to restore my other data.

 

I have excluded the Windows and Program Files directories from the backup. Why bother?

 

The registry collects clutter indefinitely, and I think it's a good idea to wipe out and rebuild your disk every year or two anyway.

 

 

Link to comment
Share on other sites

That works for you, and for me, for that matter, but very few non-technical people I know would feel comfortable reinstalling to bare metal. A backup program should be able to restore to bare metal.

 

 

 

It particularly should be able to restore OS & apps if it claims it can in its marketing. And the original poster had another point: He's a developer and could lose a lot of work in progress (and evidently did) when SFNs weren't restored.

 

 

 

Also, Windows 2000 & XP have not, in my experience (I'm a computer consultant) needed a reinstall after 18-24 months. With Windows 3.x and 9x there was no escaping it!

 

 

 

Jeff

Link to comment
Share on other sites

"I wouldn't consider using a backup program to restore installed executables or DLL's."

 

I submit that you only think that way because you're used to backup programs which don't handle this correctly. On various Unix/Linux systems I've worked with, it's perfectly reasonable to completely restore from a backup and have everything work, provided most of the hardware hasn't changed. I've even done this successfully on Windows machines if I restored from a disk image, since that correctly restores short names. But I haven't been able to successfully restore a Windows machine from a backup if I wasn't using a disk image. I didn't realize why until I saw this thread about not restoring short filenames. Until then I just thought it was yet another of Windows' quirks; after a restore some things just wouldn't work right.

 

I do agree though that it's good to reinstall Windows occasionally anyway, to clean up all the problems and clutter which seem to accumulate the longer a system is used.

Link to comment
Share on other sites

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...