Mysterious environmental variable

[colinw]

Whenever I run Ranger a strange environmental variable appears in ENV: that cannot be removed after I quit Ranger. The variable constists of a "1" followed by 106 "z" (i.e. 1zzzzzzzzzzzzz ...). The variable is empty. I can't delete the variable or remove it with Unsetenv. Is this a bug in the ENV device?

What filesystem do you have hosting the ENVARC: assignment. ?

[tonyw]

@xenic:

It's been fixed in all the shell commands?

[I'll get this response sensible some time soon]
I don't know if there are any commands or other components that are limited to 108 characters. I was using the C:Echo command and that doesn't seem to be limited to 108 characters. There are probably some that can not support 256-character file names.

But the limitation in this case is the old env-handler filesystem.

[thomasrapp]

I was using the C:Echo command and that doesn't seem to be limited to 108 characters. There are probably some that can not support 256-character file names.

Of course Echo has no file name limit because it does not access a file name anywhere. If you use > to redirect stdout to a file, this is a feature of Shell and not of the command. So you proved that Shell does not have a file name length limit.

[xenic]

What filesystem do you have hosting the ENVARC: assignment. ?

I'm using SFS/2 on the partition hosting the ENVARC: assignment. However, the issue goes beyond environmental variables. I did some more testing and there does seem to be a 107 char limit somewhere but I can't tell if it's the shell redirection command or the list command that is the problem (or both?). I tried echoing some text to a file with a name consisting of 106 "a" chars with 6 "z" chars tacked on the end like this:
echo >ram:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaazzzzzz "test"
except that it is all one line in the shell window. The forum is displaying it as seperate lines.

Using the list command looks like this:
list ram:aaa#?
Directory "ram:" on Friday 15-Jun-12
aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
aaaaaaaaaaaaaaaaaaaaaaaaaaaaz 5 ----rwed Today 11:29:35
1 file - 5 bytes - 2 blocks used
Notice that the filename is truncated to 107 chars in the list output.

Even though filesystems are capable of 256 chars, it appears that AmigaDOS is not (or some AmigaDOS commands).

EDIT: Sorry, I guess I'm really confused. IDOS-FileSystemAtt() is telling me that RAM: and my FFS & SFS/02 partitions have a maximum filename length of 107. I can create a file with a 107 char filename but AmigaDOS "Delete" won't delete the file; it just reports "object not found". If the most common filesystems have maximum filename size of 107 then why does newlib define the max filename size as 255 (NAME_MAX, FILENAME_MAX). I've seen programs use NAME_MAX or FILENAME_MAX to define the string sizes for filnames but it looks like that will just cause confusion.

[nbache]

Of course Echo has no file name limit because it does not access a file name anywhere. If you use > to redirect stdout to a file, this is a feature of Shell and not of the command. So you proved that Shell does not have a file name length limit.

Except if Tony used the TO argument, I guess?

Best regards,

Niels

[tonyw]

Thomas is right, I used the redirect character ">". I think DOS handles that, but it doesn't matter.

C:List has a problem that I've now identified - it can't handle filenames longer than 107 chars and doesn't check for name buffer overflow. I'll write a BZ about it and possibly fix it myself.

Both env-handler and ram-handler have been rewritten and now support 255-character filenames. Neither is yet ready for release. AFAIK the old versions only supported 107-character filenames and clearly there was a bug in the old env-handler. It's not worth fixing.

Trevor wrote a summary of filesystem attributes some time ago, but I don't remember where it was. That summary probably included filename length limitations.

[xenic]

@tonyw

C:List has a problem that I've now identified - it can't handle filenames longer than 107 chars and doesn't check for name buffer overflow. I'll write a BZ about it and possibly fix it myself.

C:Dir probably has the same problem. In fact, I would hope that all C: commands and system files would be tested to be sure they can handle 255 char filenames.

Both env-handler and ram-handler have been rewritten and now support 255-character filenames. Neither is yet ready for release. AFAIK the old versions only supported 107-character filenames and clearly there was a bug in the old env-handler. It's not worth fixing.

I just checked the filename size limits on all my filesystems with the IDOS->FileSystemAttrTags() function with these results:
FFS, SFX/02 & JXFS have a filename size limit of 107.
CrossDosFileSystem has a filename size limit of 106. (Copying 107 char filename to FAT volume gets truncated?)
Apparently none of our current filesystems handle 255 char filenames. If ram-handler is updated to handle 255 char filenames I would assume that we will get a new disk filesystem that will handle 255 char filenames. It would be sort of dumb to let the user create files in RAM: with filenames greater than 107 chars, only to have them truncated when they are copied to the hard-disk.

[colinw]

Apparently none of our current filesystems handle 255 char filenames. If ram-handler is updated to handle 255 char filenames I would assume that we will get a new disk filesystem that will handle 255 char filenames. It would be sort of dumb to let the user create files in RAM: with filenames greater than 107 chars, only to have them truncated when they are copied to the hard-disk.

The ram-handler and env-handler are memory based filesystems, the data block structure can be
changed at any time without problems, it's just a slightly larger memory allocation to hold longer names
and comments.

The new handlers simply reflect the new DOS filesystem interface standard that should be employed by all
filesystems in the future, when that actually happens is up to the individual programmers and their resources
to make these changes happen, but crippling the new design is not remotely appropriate to move forward,
it only provides a reason to continue to do nothing.

The env-handler being a hosted filesystem, will always reflect the limitations of any host when queeried,
that's why when you do an IDOS->FileSystemAttr() of the new handler, the call is forwarded to the host,
so you will find that you will get ENV: showing up as an SFS, JXF or FFS volume with the hosts limitations,
even though the env-handler itself can handle 255 character names and comments and 64 bit file sizes,
it is important that such limitations are not actually imposed by the env-handler itself.

Disk based filesystems are another story, the names are stored on the media itself and as such the disk block
structure may need to change to hold the extra space, this can cause compatiblity issues with previous
dostypes, such as when FFS went from 32 char names to long names.

How individual filesystems make this change is up to the programmer.

[xenic]

@colinw
Thanks for the clarification. It wouldn't hurt to have a similar explanation in the Autodocs. It might also be helpful to mention the filename length limits for the various OS4 filesystems or define them in the includes (e.g. MAX_FFS_FILENAME). It's not likely to cause problems but it would make it clear that the limits defined in newlib (255) don't necessarily apply to current filesystems.

[colinw]

Which is exactly why I wrote the IDOS->FileSystemAttr() function.
You will never know what any given filesystem will support unless you ask it.

We can always aim for a target, which is why I have included the new values
as defines in the next SDK includes (in dos/dos.h)