Anyone know what the name of the ASCII compression algorithm that is based upon 6bit ASCII and most frequently used characters?

etaonirshdl

I have working code in my old Tosser but not sure what this is called, and if it was NNTP only or if Fido supported it also?

Also, I know in the ol' days we used ROT13 to mess with n00bs (myself included)... is it still supported?

This Usenet Reader has it as a feature - and was curious anyone else have it too?

--
.. Ozz Nixon
... Author ExchangeBBS (suite)
.... Since 1983 BBS Developer

--- FMail-W32 2.0.1.4
* Origin: ExchangeBBS WHQ (1:275/362.0)

On 2019-03-28 18:34:43 +0000, Ozz Nixon -> All said:

Nalbar xabj jung gur anzr bs gur NFPVV pbzcerffvba nytbevguz gung vf onfrq hcba 6ovg NFPVV naq zbfg serdhragyl hfrq punenpgref?

rgnbavefuqy

V unir jbexvat pbqr va zl byq Gbffre ohg abg fher jung guvf vf pnyyrq, naq vs vg jnf AAGC bayl be vs Svqb fhccbegrq vg nyfb?

Nyfb, V xabj va gur by' qnlf jr hfrq EBG13 gb zrff jvgu a00of (zlfrys vapyhqrq)... vf vg fgvyy fhccbegrq?

Guvf Hfrarg Ernqre unf vg nf n srngher - naq jnf phevbhf nalbar ryfr unir vg gbb?


--
.. Ozz Nixon
... Author ExchangeBBS (suite)
.... Since 1983 BBS Developer

--- FMail-W32 2.0.1.4
* Origin: ExchangeBBS WHQ (1:275/362.0)

On 2019 Mar 28 13:37:44, you wrote to you:

Nyfb, V xabj va gur by' qnlf jr hfrq EBG13 gb zrff jvgu a00of (zlfrys vapyhqrq)... vf vg fgvyy fhccbegrq?

vg vf fgvyy fhccbegrq ohg vg qrcraqf ba gur ernqre fbsgjner hfrq ;)

)\/(ark

Always Mount a Scratch Monkey
Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
... Some settling of files may occur during ZIPment...
---
* Origin: (1:3634/12.73)

Uryyb Bmm.

28 Zne 19 13:34, lbh jebgr gb Nyy:

Guvf Hfrarg Ernqre unf vg nf n srngher - naq jnf phevbhf nalbar ryfr
unir vg gbb?

TbyqRq/TbyqRq+ fhccbegf EBG-13.

Yngre,
Frna

--- GoldED/2 3.0.1
* Origin: Outpost BBS - bbs.outpostbbs.net (1:18/200)

Great to know... in your JAM implementation, what theory(s) did you implement for MSG_COMPRESS and MSG_ENCRYPT?

In your implementation, what are you using for the MSGID 1:3634/12.73 5c9e81b1 <-- 5c9e81b1?

Ozz

--
.. Ozz Nixon
... Author ExchangeBBS (suite)
.... Since 1983 BBS Developer

--- FMail-W32 2.0.1.4
* Origin: ExchangeBBS WHQ (1:275/362.0)

On 2019 Mar 29 18:40:52, you wrote to me:

Great to know... in your JAM implementation, what theory(s) did you implement for MSG_COMPRESS and MSG_ENCRYPT?

none... i never messed with it because it wasn't really documented and no one else was doing anything with it... RA and FD never implemented anything to have as a bed to compare against...

In your implementation, what are you using for the MSGID 1:3634/12.73 5c9e81b1 <-- 5c9e81b1?

i have no idea what is being used... i didn't write GoldED and i haven't dug into the C code to try to find out...

)\/(ark

Always Mount a Scratch Monkey
Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
... May you never remember these curses when you try to.
---
* Origin: (1:3634/12.73)

Hey mark!

i didn't write GoldED and i haven't dug into the C code to try
to find out...

Just looking at it I get the impression that it is the 32 bit hex value for unixtime. Take a look at the following for confirmation;

TZ=EST5EDT date --date=@$(printf "%d" 0x5c9ec143) +"%d %b %y %T" = 29 Mar 19 21:07:15

which is very close to the obsoleted msgHeader's datetime stamp in your reply = 29 Mar 19 21:04:38.

Coincidence? I think not. I am guessing golded adds the MSGID just before you save it given there is just under 3 minutes difference. In the routine I use it creates the MSGID from the same unixtime as the obsoleted msgHeader's datetime stamp and thus are identical.

Life is good,
Maurice

... Don't cry for me I have vi.
--- GNU bash, version 5.0.3(1)-release (x86_64-pc-linux-gnu)
* Origin: Little Mikey's EuroPoint - Ladysmith BC, Canada (2:280/464.113)

On 2019-03-30 06:22:44 +0000, Maurice Kinal -> mark lewis said:

Maurice,

Just looking at it I get the impression that it is the 32 bit hex value
for unixtime. Take a look at the following for confirmation;

TZ=EST5EDT date --date=@$(printf "%d" 0x5c9ec143) +"%d %b %y %T" =
29 Mar 19 21:07:15

which is very close to the obsoleted msgHeader's datetime stamp in your
reply = 29 Mar 19 21:04:38.

Thank you *very* much... I was hesitant to using timestamp - however, it would make sense on the server to use timestamp - and a piece of logic to make sure no two nodes post during the same second. [shooting for being popular in the future *g*]...

O.

--
.. Ozz Nixon
... Author ExchangeBBS (suite)
.... Since 1983 BBS Developer

--- FMail-W32 2.0.1.4
* Origin: ExchangeBBS WHQ (1:275/362.0)

Hey Ozz!

it would make sense on the server to use timestamp

Agreed. There is very little chance that the sysop would be creating two messages at the same time which is the danger on a multiuser system such as a busy BB S.

two nodes post during the same second

That won't matter since the node number is part of the MSGID. Thus even if the 32 bit hex number is exactly the same the nodes' MSGID will still be unique. It is only if a multisuer node with only one address is this a danger and probably not much of a danger these days. Also a sysop could give each user on a node a point address so that MSGID uniquiness would be guarenteed ... at least up to Sun Feb 7 06:28:15 UTC 2106, for unsigned 32 bit unixtimes which from my observations seems to be the case in Fidonet, not counting abandonware of course. Your mileage may vary.

Life is good,
Maurice

... Don't cry for me I have vi.
--- GNU bash, version 5.0.3(1)-release (x86_64-pc-linux-gnu)
* Origin: Little Mikey's EuroPoint - Ladysmith BC, Canada (2:280/464.113)

Re: Name that compression...
By: Ozz Nixon to All on Thu Mar 28 2019 01:34 pm

Anyone know what the name of the ASCII compression algorithm that is based upon 6bit ASCII and most frequently used characters?

etaonirshdl

I have working code in my old Tosser but not sure what this is called, and if it was NNTP only or if Fido supported it also?

Also, I know in the ol' days we used ROT13 to mess with n00bs (myself included)... is it still supported?

This Usenet Reader has it as a feature - and was curious anyone else have it too?

Look for xananews, open source. I think it has it if looking for code samples.
xxcarol

PS: back from work trip
--- SBBSecho 2.12-Win32
* Origin: SHENK'S EXPRESS, shenks.synchro.net (1:275/100)

Re: Re: Name that compression...
By: Ozz Nixon to Ozz Nixon on Thu Mar 28 2019 01:37 pm

Nalbar xabj jung gur anzr bs gur NFPVV pbzcerffvba nytbevguz gung vf onfrq hcba 6ovg NFPVV naq zbfg serdhragyl hfrq punenpgref?

Geshsundheight!
;-)
carol
--- SBBSecho 2.12-Win32
* Origin: SHENK'S EXPRESS, shenks.synchro.net (1:275/100)

On 2019-03-30 08:36:24 +0000, Maurice Kinal -> Ozz Nixon said:

Also a sysop could give each user on a node a point address so that
MSGID uniquiness would be guarenteed ... at least up to Sun Feb 7
06:28:15 UTC 2106, for unsigned 32 bit unixtimes which from my
observations seems to be the case in Fidonet, not counting abandonware
of course. Your mileage may vary.

Brilliant! Off to code that each node posts as a point - I did a test with 2 users yesterday, we all 3 are on ExchangeBBS NNTP server - were able to produce 2 messages with the same MSGID. This point concept totally fixes that (Kudos!).

Ozz


--
.. Ozz Nixon
... Author ExchangeBBS (suite)
.... Since 1983 BBS Developer

--- FMail-W32 2.0.1.4
* Origin: ExchangeBBS WHQ (1:275/362.0)

Hi Ozz!

28 Mar 2019 13:37, from Ozz Nixon -> Ozz Nixon:

Anyone know what the name of the ASCII compression algorithm that is
based upon 6bit ASCII and most frequently used characters?

No.
What use would it be?

CU, Ricsi

... Whenever I feel like exercise, I lie down until the feeling passes.
--- GoldED+/LNX
* Origin: What happens if you get scared half to death twice? (2:310/31)

Hey Ozz!

This point concept totally fixes that

I believe it is the best fix thus far given that it doesn't require a radical change. However the hex part will enventually require additional nibbles as time progresses beyond 32 bits. An additional nibble will extend the shelf life to Sun Aug 20 07:32:15 UTC 4147 and as long as you don't code that it must be 32 bits then nothing will need to be done come Sun Feb 7 06:28:15 UTC 2106.

printf "%x\n" $(date --date="Sun Feb 7 06:28:16 UTC 2106" +%s) = 100000000

Note the extra nibble. Mind you the above works simply because coreutils' date command uses a 64 bit floating point for unixtime which has a shelf life to Wed Dec 31 23:59:59 UTC 2147485547 all things being equal ... which is highly unlikely over two billion years.

Life is good,
Maurice

... Don't cry for me I have vi.
--- GNU bash, version 5.0.3(1)-release (x86_64-pc-linux-gnu)
* Origin: Little Mikey's EuroPoint - Ladysmith BC, Canada (2:280/464.113)

On 2019 Mar 29 22:06:18, you wrote to Maurice Kinal:

it would make sense on the server to use timestamp - and a piece of
logic to make sure no two nodes post during the same second.

i would not rely on seconds granularity at all... sure, for manual posting but certainly not for offline QWK/QWKE/BW uploads where the user may upload 50+ messages in one go... are you going to make them wait for 50 seconds while you delay a second for each message? what happens if you have two or more users online which upload their offline replies at the same time? sure, the odds are against that happening but we all know the name of Murphy very well, don't we ;)

then there's external automated posting tools that can easily post 100+ messages per second on today's hardware...

using the timestamp along with a plain simple serial number would work... but for that matter just a plain simple serial number is really all that is needed... i never did figure out why so many tried to come up with convoluted serial number generating routines... i remember that many used CRC16 and then CRC32 which are well known to have limited range as well as easily hit collisions... then some thought of using MD5 which then had to be chopped to fit within the limits of the serial number and again, limited range and collisions reared their heads...

but i know of at least one format, written in 1994 and implemented that same year, that incorporates the multinode node number, timestamp and serial number...

==== Begin "MSGID.TXT" ====
Area: FIDONet Developers Area (E)
Date : Apr 15 '94, 20:20
From : Leonard Erickson 1:105/51.0
To : Paul Edwards
Subj : Re: EchoMail ÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ Ä

Quoting Paul Edwards to Mikael St†ldal <=-

There is no requirement for there to be a MSGID kludge, and can you honestly say that you have NEVER been at risk of violating the MSGID standard, ie there was no possibility that you ever generated two
MSGID's the same within any 3 year period? I certainly can't.

The identifier part of the MSGID is 8 hex digits. As a worst case,
there are 1096 days in 3 years. (assuming that there's aleap year in
there somewhere). There are 86400 seconds in a day. That gives 94694400 seconds. In HEX that's 5A4EC00. Note that this is *7* digits. The max
number of possible ID's is 100000000h. Doing the division gives 2Dh or
45d IDs for each second. I think we can live with a system that's
limited to tossing messages at 45/second. Or that tosses faster, but
won't let itself be run again until enough time has passed. 3.9
*million* messages per day is more traffic than anybody is likely to
generate!

As a *simple* method of doing this, have the program generate the
Julian Day Number (April 15, AD 1994 is JD#2449457). Then figure modulo
2048 of it. That's 49. Shift it to the right an appropriate number of
bits (multiply by 20 00 00 hex). That gives us 62 00 00 00 hex. And we
have 20 00 00 hex or 2,097,152 decimal message numbers per day. I think
that can be handled easily enough. Just generate a message number
(starting at 0 for the start of the day) and add it to the modified jd.

So we get:
serial = (JD# mod $800) * $200000 + mnum

That's *not* that hard. And you can even allow for multiple tasks by
just allocated the least significant digit or digits. With 2 digits
allocated for a task number, that gives each task 8k numbers to
allocate per day. I think that's enough. :-)

For up to 16 tasks:
serial = (JD# mod $800) * $200000 + num *16 + task#

For up to 256 tasks:
serial = (JD# mod $800) * $200000 + num *256 + task#

... Unrecoverable Application Error: (A)bort, (R)etry, (B)uy Desqview ?
-!- Blue Wave/QBBS v2.12 [NR]
! Origin: Shadowshack (1:105/51.0)

==== End "MSGID.TXT" ====

the above may not be sufficient, though, if one is importing usenet newsgroups and generating MSGID serial numbers for them... however, the idea is quite workable...

i have pascal code that does the above with some slight variation and at least two FTN capable programs are using my code under license but it is still simple enough to create and manage... it was testing on a live system for just over three years before being released for public consumption in utility tools...

the only real gotcha is if the incremental serial number file is lost... then it may be possible to generate duplicate serial numbers on the same day from the same node but once the date rolls over, that is taken care of... unless the system clock gets reset for some reason... ok, so that's two possible gotchas...

either way, plain straight incremented numbers are the easiest and you don't need to worry about the clock or how many nodes... just that the serial number file is not reset... of course proper file locking and access is needed to protect against two or more nodes getting the same number at the same time...

so anyway, there it is, FWIW...

)\/(ark

Always Mount a Scratch Monkey
Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
... Too many people think a potato tastes like a Pringle.
---
* Origin: (1:3634/12.73)

On 2019 Mar 30 03:36:24, you wrote to Ozz Nixon:

it would make sense on the server to use timestamp

Agreed. There is very little chance that the sysop would be creating
two messages at the same time which is the danger on a multiuser
system such as a busy BB S.

this horizon is too limited... like yourself, too many think only of manually posted messages and do not take into account offline mail users or automated posting tools like those used to post monthly rules or advertisements or even newly arrived files listings...

two nodes post during the same second

That won't matter since the node number is part of the MSGID.

wrong node number, man... he's talking about a multinode BBS where you can have more than one user online at the same time...

Thus even if the 32 bit hex number is exactly the same the nodes'
MSGID will still be unique.

true...

It is only if a multisuer node with only one address is this a danger
and probably not much of a danger these days.

you might be surprised ;)

Also a sysop could give each user on a node a point address

that would make them points and point software would handle that because why would a user with a point address connect to a BBS to pull their mail manually when they can just use the mailer in the point software and poll for the mail like any other FTN system? said mail could even be delivered by their host if they leave their point mailer running all the time and the necessary connection data is available to the host...

however, along this line of using point addresses, a multinode system could use points for each of their nodes... we used to do that way back when we first got into fidonet... we were running two nodes of RemoteAccess and FrontDoor under DESQview on a 286... that was before (shareware) FD could do multinode... i even published a paper on how to do it complete with BBS door examples... the paper was called MULTNODE at that time... each mailer node shared the same outbound and the tosser was set up as two separate tossers with the point address for that node/task... the same for the BBS but they shared the userbase, message base and files base along with the external doors... it was a fun configuration to work with O:)

so that MSGID uniquiness would be guarenteed ... at least up to Sun
Feb 7 06:28:15 UTC 2106, for unsigned 32 bit unixtimes which from my observations seems to be the case in Fidonet, not counting abandonware
of course. Your mileage may vary.

it comes much sooner for systems using signed long integers like those from the TP/BP5,6,7 days...

2038 Jan 19 03:14:07 UTC

then we have the NTP overflow problem with its use of 64bits which comes two years earlier on about 2036 Feb 07 06:28:15 UTC... this problem comes because 32bits are used for seconds since 1900 Jan 1 and the other 32bits are used for fractional seconds...

i wish i had some c0ffee :(

)\/(ark

Always Mount a Scratch Monkey
Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
... Total is $1000. $10 for the upgrade, and $990 s/h.
---
* Origin: (1:3634/12.73)

On 2019 Mar 28 13:34:42, you wrote to All:

Anyone know what the name of the ASCII compression algorithm that is based upon 6bit ASCII and most frequently used characters?

are you thinking of uuencode or xxencode?

)\/(ark

Always Mount a Scratch Monkey
Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
... "Even the best of friends cannot attend each other's funeral." Ä Albran
---
* Origin: (1:3634/12.73)

Hey mark!

do not take into account offline mail users

True but in that case the incremental methodology you posted could be deployed for robotic packing. Now if offline readers generated their own MSGIDs then this wouldn't be an issue. Either that or get rid of MSGIDs but I already know of one well used messaging system that will freak out and cause much grief. I won't say which one.

or automated posting tools like those used to post monthly rules
or advertisements or even newly arrived files listings...

Why wouldn't they be generating their own MSGID?

wrong node number, man

Ah! Too late now.

you might be surprised ;)

Yes I would be.

it comes much sooner for systems using signed long integers like
those from the TP/BP5,6,7 days...

I am guessing their definition of long ints was 32 bits given the shelf lifes you posted for ntp and unixtime. These days it isn't an issue and if it is then there is another good reason for abandoning abandonware. If I am not mistaken ntp has made the switch to 64 bit ints quite some time ago. Speaking of which, beats me why they'd need a 64 bit int for fractional seconds. Heck even 32 bit fractional second is overkill other than for nanoseconds which are way beyond the resolution of most computer clocks and network speeds. The best error I've seen posted for clocks was in the order of 30-40 nanoseconds and we're unlikely to ever see such a clock in action.

Life is good,
Maurice

... Don't cry for me I have vi.
--- GNU bash, version 5.0.3(1)-release (x86_64-pc-linux-gnu)
* Origin: Little Mikey's EuroPoint - Ladysmith BC, Canada (2:280/464.113)

On 2019 Mar 30 16:40:22, you wrote to me:

do not take into account offline mail users

True but in that case the incremental methodology you posted could be deployed for robotic packing.

this is true and when it really comes down to it, all FTN related messaging tools should use the same serial number generating routines to avoid separate programs generating the same serial number...

Now if offline readers generated their own MSGIDs then this wouldn't
be an issue.

they cannot... maybe QWKE but the readers on the user's end just don't... remember that this QWK/QWKE/BW stuff was porce-pushed into FTNs by folks coming from the RIME/PCRelay side of the world...

Either that or get rid of MSGIDs but I already know of one well used messaging system that will freak out and cause much grief. I won't
say which one.

hehehe... i'd say that one would be better off to develop new formats of packets and packed messages to get rid of these old problems... however, without more developers, it just won't happen... then there's the historic side where folks want to bring their old systems back out of hibernation now that they've had years to play on the 'net and find out it really isn't all that to begin with ;)

or automated posting tools like those used to post monthly rules or
advertisements or even newly arrived files listings...

Why wouldn't they be generating their own MSGID?

why wouldn't what be doing that?

wrong node number, man

Ah! Too late now.

hehehe...

you might be surprised ;)

Yes I would be.

:)

it comes much sooner for systems using signed long integers like
those from the TP/BP5,6,7 days...

I am guessing their definition of long ints was 32 bits

in pascal back in the day, you had byte, word, real, int and longint... longint was signed 32bit... these days, there are more available and even unsigned ones... word would have worked but it was only two bytes... it there had been a 32bit word, things would have been a little different... they call it a longword, these days...

FWIW: From the FPC manual, this is the new stuff available today but you can see the ranges of what has been available since the beginning...

integer types
Type Range Bytes
=========================================================
Byte 0 .. 255 1
Shortint -128 .. 127 1
Smallint -32768 .. 32767 2
Word 0 .. 65535 2
Integer smallint or longint 2 or 4
Cardinal longword 4
Longint -2147483648 .. 2147483647 4
Longword 0..4294967295 4
Int64 -9223372036854775808 .. 9223372036854775807 8
QWord 0 .. 18446744073709551615 8

Free Pascal does automatic type conversion in expressions where different kinds of integer types are used.

real types
Type Range Significant digits Bytes
=========================================================
Real platform dependent ??? 4 or 8
Single 1.5E-45 .. 3.4E38 7-8 4
Double 5.0E-324 .. 1.7E308 15-16 8
Extended 1.9E-4932 .. 1.1E4932 19-20 10
Comp -2E64+1 .. 2E63-1 19-20 8
Currency -922337203685477.5808 922337203685477.5807 8

given the shelf lifes you posted for ntp and unixtime. These days it isn't an issue and if it is then there is another good reason for abandoning abandonware.

yep, and the reasons for doing so are getting more nad more prolific, too...

If I am not mistaken ntp has made the switch to 64 bit ints quite some time ago.

yes but there's still a lot of older embedded systems in place... they'll be found when they fall over and/or start emitting invalid dates...

Speaking of which, beats me why they'd need a 64 bit int for
fractional seconds. Heck even 32 bit fractional second is overkill
other than for nanoseconds which are way beyond the resolution of most computer clocks and network speeds. The best error I've seen posted
for clocks was in the order of 30-40 nanoseconds and we're unlikely to ever see such a clock in action.

i dunno... i suspect it was for extremely accurate timing capabilities... maybe something to do with nuclear isotope timings?? one would have to check the history to see for sure...

)\/(ark

Always Mount a Scratch Monkey
Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
... Chocolate and <FN>.....my favorite Sweets! ;*)
---
* Origin: (1:3634/12.73)

Hey mark!

to avoid separate programs generating the same serial number...

That can be dealt with by assigning points to the different programs. Using the offline messaging example, if the converter from user to BBS has it's own point address and increments from it's start up time, then there will never be any duplication of MSGIDs, they will always be unique even if the 8 nibble hex string is exactly the same as your Golded MSGID which I note is using 1:3634/12.73, which we've already determined is using unixtime to generate the 8 nibble hex string. For argument's sake, say you offline packer uses 1:3634/12.74 and happens to coinicide with 5c9fbf47 while assigning an 8 nibble hex string to a message in an incoming offline packet, then the resulting MSGID will be "MSGID: 1:3634/12.74 5c9fbf47" instead of ""MSGID: 1:3634/12.73 5c9fbf47". Also it should never collide with any other program's MSGID if they all have their own point address even when the initial hex string is identical and therefore the following incremented hex strings.

maybe QWKE but the readers on the user's end just don't

Another reason I never got into offline readers. What I am currently doing is definetly offline messaging but the messages generated are a stripped down version of the packed MSG format with a single unixtime used to generate the obsolete ftn datetime stamp as well as a hex string for the MSGID. It is very nice.

in pascal back in the day, you had byte, word, real, int and
longint

I am uncertain as to when "back in the day" was. However when I first learned F77 way back when, an int was 32 bits, whether signed or unsigned. What Pascal is calling an int would be called a short (16 bits) and they could also be signed or unsigned. If I recall correctly, long could be either 32 bits or 64 bits depending on the platform and it seems to me the on the VAX/VMS system I was working on back then, it was 64 bits. Most of the stored data (9 track tape) were 32 bit floats with an ASCII 512 byte header for individual records or files or whatever the kids these days are calling them. Compression was (is?) definetly vorbotten. Also all datetime stamps were in UTC. That was all that mattered.

there's still a lot of older embedded systems in place

I have absolutely no doubt about that.

i suspect it was for extremely accurate timing capabilities

For sure but if the error is over 30 times the resolution (1ns) then extremely accurate is out the window. I am not sure what the resolution for modern atomic clocks might be but I am sure that neither of us has direct access to them and even if we did we couldn't take advantage of the increased accuracy. I certainly don't have anything handy that could even come close.

Life is good,
Maurice

... Don't cry for me I have vi.
--- GNU bash, version 5.0.3(1)-release (x86_64-pc-linux-gnu)
* Origin: Little Mikey's EuroPoint - Ladysmith BC, Canada (2:280/464.113)

On 2019-03-30 16:03:02 +0000, Richard Menedetter -> Ozz Nixon said:

Hi Ozz!

28 Mar 2019 13:37, from Ozz Nixon -> Ozz Nixon:

ON> Anyone know what the name of the ASCII compression algorithm that is
ON> based upon 6bit ASCII and most frequently used characters?

No.
What use would it be?

Actually it is *very* useful in the 7bit ASCII world of Fidonet messages. Or other #32..#127 character based strings. There are 2 I have looked at - the algorithm I already have beats shoci, the other is called smaz (which beats this algorithm). I am porting smaz to Pascal, however, the algorithm I have is very simple to read and understand - whereas, smaz is doing post compression matching of common patterns to make it even smaller output.

I am just curious if anyone knows what the heck this algorithm is called. I found it and used it in my tosser in 1994 - thus where I came across it recently.

If you google compression etaonirshdl you will find someone has ported it to Delphi. I found this with a 1996 copyright on it: https://github.com/dblock/xreplace/blob/master/Classes/XReplace/lzh.pas - which is not LZH in any way.

What use would it be? Simple storage compression in JAM for my NNTP Server I wrote (working on the web interface now). MODE STREAM from machine to machine in real-time - simple compressions like these can be much more efficient than dictionary based algorithms, and much smaller memory footprint - when streaming millions of messages a day from usenet -> my server in AZ -> my server in TX (and) -> to my server in VA which is moving to FL in 14wks +/-.

Ozz


--
.. Ozz Nixon
... Author ExchangeBBS (suite)
.... Since 1983 BBS Developer

--- FMail-W32 2.0.1.4
* Origin: ExchangeBBS WHQ (1:275/362.0)

On 2019-03-30 16:21:38 +0000, mark lewis -> Ozz Nixon said:

On 2019 Mar 28 13:34:42, you wrote to All:

ON> Anyone know what the name of the ASCII compression algorithm that
is based
ON> upon 6bit ASCII and most frequently used characters?

are you thinking of uuencode or xxencode?

+.5 for effort! ;-)

no, if you *really* want to get into it - search short string compression. A very popular request across many languages and projects.


--
.. Ozz Nixon
... Author ExchangeBBS (suite)
.... Since 1983 BBS Developer

--- FMail-W32 2.0.1.4
* Origin: ExchangeBBS WHQ (1:275/362.0)

On 2019-03-30 18:54:00 +0000, mark lewis -> Maurice Kinal said:

integer types
Type Range Bytes
=========================================================
Longint -2147483648 .. 2147483647 4
Longword 0..4294967295 4

FYI ... this is why PCBoard used Microsft C's Single... signed long broke at 2.1 billion. Masking it to Single, allowed them to exceed it - however, translators for other compilers could move bytes around from MS-Single -> Pascal Real, but with a cap of 16,700,000 [as MSB defaulting to set in MS C's storage limited it to 16.7 million].

Now, a trick I have been doing in the 16bit compiler is using 8 byte [double] on disk storage - it is bit for bit compatible with 32bit compiler's Int64 (signed). [again MSB is not compatible - but ignoring it - I have ported PCB RIME feeds for 18 years into a single file without corruption. But, RIME died when CDC screwed everyone...

Free Pascal does automatic type conversion in expressions where
different kinds of integer types are used.

real types
Type Range Significant digits Bytes
=========================================================
Real platform dependent ??? 4 or 8
Single 1.5E-45 .. 3.4E38 7-8 4
Double 5.0E-324 .. 1.7E308 15-16 8
Extended 1.9E-4932 .. 1.1E4932 19-20 10

FYI - talking with Florian and Jonas, Extended is not supported on Windows 10 64bit (per Microsoft). * I could not find reference to it, but, if Florian tells me something of this nature, I accept it. I had to redesign a lot of things that used to support Extended.

Ozz

--
.. Ozz Nixon
... Author ExchangeBBS (suite)
.... Since 1983 BBS Developer

--- FMail-W32 2.0.1.4
* Origin: ExchangeBBS WHQ (1:275/362.0)

On 2019-03-31 00:28:39 +0000, Maurice Kinal -> mark lewis said:

I am uncertain as to when "back in the day" was. However when I first learned F77 way back when, an int was 32 bits, whether signed or
unsigned. What Pascal is calling an int would be called a short (16
bits) and they could also be signed or unsigned.

(*) Clarity - "Turbo Pascal", not "Pascal". Turbo Pascal 6.0->7.0.1 treated "Integer" as 16bit unsigned, however, BP7 & 8, TPW, Delphi, FPC and Modern Pascal, all define "Integer" as 32bit. Same problem C guys have when picking up some of the old code "int" 16bit or 32bit? And lastly, Turbo Pascal had "shortint" as signed 8bit - I always add the type smallint in my old projects - and s/Integer/SmallInt/gi as soon as I open old code.


--
.. Ozz Nixon
... Author ExchangeBBS (suite)
.... Since 1983 BBS Developer

--- FMail-W32 2.0.1.4
* Origin: ExchangeBBS WHQ (1:275/362.0)

Hey Ozz!

(*) Clarity - "Turbo Pascal", not "Pascal".

To me it is a pot-eh-toe/pot-ah-toe issue as I never had any experience with pascal. I did look at the Free Pascal not too long ago at mark's prompting but that is as far as I ever went with it. Seems to me it had something to do with a certain BBS project.

Same problem C guys have when picking up some of the old code
"int" 16bit or 32bit?

Sorry but when I picked up C around 1989-ish an int was 32 bit and 16 bit ints were called shorts. I briefly used Borland C for DOS and if I am not mistaken there was this issue but not for long as I switched to a 32 bit compiler for DOS just to stay more or less in sync with Unix based systems. DJGPP if I recall correctly. That was for about 3 or 4 years and then I started using Linux on PC's which solved any compatibilty issues I had thanks to gcc. The only things I ended up porting from DOS was some f77 based code and I will NEVER do that again. That was about 1996-ish and any C based code I had on DOS got completely abandoned. Linux had removed the need for me to write C code to make DOS more Unixie.

That's my story and I'm sticking to it.

Life is good,
Maurice

... Don't cry for me I have vi.
--- GNU bash, version 5.0.3(1)-release (x86_64-pc-linux-gnu)
* Origin: Little Mikey's EuroPoint - Ladysmith BC, Canada (2:280/464.113)

Re: Name that compression...
By: Maurice Kinal to Ozz Nixon on Sat Mar 30 2019 03:47 pm

date command uses a 64 bit floating point for unixtime which has a shelf life to Wed Dec 31 23:59:59 UTC 2147485547 all things being equal ... which is highly unlikely over two billion years.

Yeah, I will have to remember to login and check my software the next morning to see if I broke or not :-)
--- SBBSecho 2.12-Win32
* Origin: SHENK'S EXPRESS, shenks.synchro.net (1:275/100)

Hey Ozz!

Yeah, I will have to remember to login and check my software the
next morning to see if I broke or not :-)

Judging by the message I am replying to I am guessing there may be an issue.

Life is good,
Maurice

... Don't cry for me I have vi.
--- GNU bash, version 5.0.3(1)-release (aarch64-raspi3b+-linux-gnu)
* Origin: Little Mikey's CanadARM - Ladysmith BC, Canada (1:153/7001.2989)

Re: Name that compression...
By: Maurice Kinal to Ozz Nixon on Fri Apr 12 2019 12:44 pm

Yeah, I will have to remember to login and check my software the
next morning to see if I broke or not :-)

Judging by the message I am replying to I am guessing there may be an issue.

Grin, I was out dealing wth some home issues but Ozz was working hard and warned me he had his feed piling up for a bit. He does that when he works on things that have the potential of causing dups and such. I have a 16 hour backlog just now.

Not an issue but when he's working, he tends to incognito a bit ;-)

xxcarol
--- SBBSecho 2.12-Win32
* Origin: SHENK'S EXPRESS, shenks.synchro.net (1:275/100)

Hey Carol!

Not an issue but when he's working, he tends to incognito a bit

Ah! That would explain it then.

Life is good,
Maurice

... Don't cry for me I have vi.
--- GNU bash, version 5.0.3(1)-release (aarch64-raspi3b+-linux-gnu)
* Origin: Little Mikey's CanadARM - Ladysmith BC, Canada (1:153/7001.2989)