Hello Ozz.

08 Feb 19 22:52, I wrote to Wilfred van Velzen:

FMail to me ECHOLIST with new posts via TOSS. And what is the best way

Realized that could also mean "NEW ECHO LIST"... but, mainly anything to shortcircuit time a BBS needs for "whats new" for users who actively read and write in the echos. I can then accumulate a list of ECHOS by DAY - and when a user logs back in - POOF there is almost real-time list of new ECHO POSTS, and I can use that for quickly FindNewByUser(UserCRC, UserID) which can pull from LastRead - or I could pre-create a list - all about making QuickBBS quicker...

Again, thanks! Keep up the work! I have racks of servers with different OSes and APPs if you need a test bed, let me know. (I am good at finding bugs, and making them too) ;-)

Ozz

--- FMail-W32 2.0.1.4
* Origin: Richmond VA (RVA) Fidonet Support (1:275/362)

Hi Ozz,

On 2019-02-08 22:52:26, you wrote to me:

Firstly, KUDOS! Unzip 2.0 over 1.59b

"Unzip"? ;)

- run FTOOLS MAINT (AWESOME REPLY CHAIN FEATURE!!!!!) ... then ran the Config EXE - everything aligned perfectly, no wacky characters in
fields or anything that other packages do - AWESOME UPGRADE!

Did the 1.59b do that? I don't think I ever used that beta version. The source I got from the original author was for 1.60. I did quite a bit of tweaking/improving/fixing/speeding-up on the maint code, but it's still based on the original by Folkert, so kudos must go to him too! ;)

Per RETRO copies: http://archives.thebbs.org/ra53a.htm

Thanks.

I am developing a new JAMAPI (TP version had a few quirks - no
longer). I have also modernized it to current class/object structure.
I will be incorporating your "Reply Chain" feature into the JAMAPI (I
call DXJAMAPI to avoid confusion).

Well good luck with the porting from C to Pascal! ;)

* Only major wishlist I did not see (may be there) -- a drop file from FMail to me ECHOLIST with new posts via TOSS.

You mean something like what is in the toss log file but in a more machine readable form?

And what is the best way for me to pass such from BBS/NNTP/etc to you
for SCAN?

Ehhrrr?

Bye, Wilfred.

--- FMail-lnx64 2.1.0.18-B20170815
* Origin: FMail development HQ (2:280/464)

Hi Ozz,

On 2019-02-08 23:04:26, you wrote to you:

FMail to me ECHOLIST with new posts via TOSS. And what is the best
way

Realized that could also mean "NEW ECHO LIST"... but, mainly anything to shortcircuit time a BBS needs for "whats new" for users who actively read and write in the echos. I can then accumulate a list of ECHOS by DAY - and when a user logs back in - POOF there is almost real-time list of new ECHO POSTS, and I can use that for quickly FindNewByUser(UserCRC, UserID) which can pull from LastRead - or I could pre-create a list - all about making QuickBBS quicker...

Isn't that what the .jlr files in the JAM message base are for? They contain the last read "pointers" for each user, and they are updated everytime fmail updates the jam area files, as far as I remember...?

Again, thanks! Keep up the work! I have racks of servers with
different OSes and APPs if you need a test bed, let me know. (I am
good at finding bugs, and making them too) ;-)

Currently I and the single other test node, only use FMail with Binkly Style Outbound (BSO), and Golded as editor. And no bbs. So it could use some testing in other environments...

Bye, Wilfred.

--- FMail-lnx64 2.1.0.18-B20170815
* Origin: FMail development HQ (2:280/464)

Hello Wilfred.

12 Feb 19 21:42, you wrote to me:

Hi Ozz,

I am developing a new JAMAPI (TP version had a few quirks - no
longer). I have also modernized it to current class/object
structure. I will be incorporating your "Reply Chain" feature
into the JAMAPI (I call DXJAMAPI to avoid confusion).

Well good luck with the porting from C to Pascal! ;)

I do it all day ;-)

* Only major wishlist I did not see (may be there) -- a drop file
from FMail to me ECHOLIST with new posts via TOSS.

You mean something like what is in the toss log file but in a more
machine readable form?

For example, d'Bridge tosser would let me know
FTSC_PUBLIC
FMAIL_HELP
FN_SYSOP

Which my BBS used to circumvent a full scan of 1000+ echos when someone would log in, that was on before the most recent toss - I limited their "NEW SCAN" to those 3 echos listed above. I have a small DB of userID,array of areas - so it only searched the array of areas from JLR forward. *MUCH* faster for a user on my systems... especially when I start pulling in newsgroups and any other FTN(etwork) I can get coming in.

And what is the best way for me to pass such from BBS/NNTP/etc to
you for SCAN?

Same going out, I could drop SCANTHIS.LST and simply drop in:
AREA or AREA,MSG#

To short-circuit scanning for timestamp or modcounter changes in JAM.

Ozz

--- FMail-W32 2.0.1.4
* Origin: Richmond VA (RVA) Fidonet Support (1:275/362)

Hello Wilfred.

12 Feb 19 21:51, you wrote to me:

Hi Ozz,

Isn't that what the .jlr files in the JAM message base are for? They contain the last read "pointers" for each user, and they are updated everytime fmail updates the jam area files, as far as I remember...?

Yes, but hitting 1000+ JLR files all over a drive *is slow* versus, SCANTHIS.LST from me to you - you only hit 5 to 10 echos that have been active since I signaled FMAIL SCAN.

And vise versa, FMAIL and Rhenium run in a seperate thread/window from the BBS which is actually running right now behind GameSrv.exe in dynamic windows. * This will *SERIOUSLY* change when I migrate this to my Linux rack. However, in either case, I do not "SCAN" after every caller, and I do not "TOSS" after every mailer session. I am currently doing it on the 5's (every 5th minute I check if SCANTHIS.LST exists then I do a FMAIL SCAN) and if /BinkPDOS/IN/ has anythign other than . and .. then I call FMAIL TOSS.

I do this, as I get 100's of port scans a minute that would normally trigger a "FMAIL TOSS" and really eat CPU... and depending upon their script - could cross-fire FMAIL SCAN if I ran it in the STARTBBS.BAT for GameSrv.exe

Currently I and the single other test node, only use FMail with Binkly Style Outbound (BSO), and Golded as editor. And no bbs. So it could
use some testing in other environments...

If you do not mind *crazy* ideas to steamline my systems across the US - I will gladly share ideas. I can even share how d'Bridge does its folders - I call DBO for my environment (which supports both BSO and DBO concurrently)... MODEM vs BINKP. I am currently looking at scheduling a rewrite on Portal of Power, and merging my BinkPDOS code into it giving me EMSI/WAZOO/BINKP all on a single codebase (DOS/WINDOWS/MAC/LINUX).

Ozz

--- FMail-W32 2.0.1.4
* Origin: Richmond VA (RVA) Fidonet Support (1:275/362)

Hi Ozz,

On 2019-02-14 10:09:08, you wrote to me:

* Only major wishlist I did not see (may be there) -- a drop file
from FMail to me ECHOLIST with new posts via TOSS.

You mean something like what is in the toss log file but in a more
machine readable form?

For example, d'Bridge tosser would let me know
FTSC_PUBLIC
FMAIL_HELP
FN_SYSOP

Which my BBS used to circumvent a full scan of 1000+ echos when someone would log in, that was on before the most recent toss - I limited their "NEW SCAN" to those 3 echos listed above. I have a small DB of

userID,array

of areas - so it only searched the array of areas from JLR forward. *MUCH* faster for a user on my systems... especially when I start pulling in newsgroups and any other FTN(etwork) I can get coming in.

There's already this option in the config:

Tossed Areas List

Writes a list of echomail areas in which mail has been
tossed to the specified file.

I don't use it myself, but it seems just what you want.

And what is the best way for me to pass such from BBS/NNTP/etc to
you for SCAN?

Same going out, I could drop SCANTHIS.LST and simply drop in:
AREA or AREA,MSG#

To short-circuit scanning for timestamp or modcounter changes in JAM.

There is the "echomail.jam" that's read by fmail scan. Golded for instance creates it.

Scan always

If set to "Yes", the messages base will always be
completely scanned for outgoing messages. If set to
"No", FMail relies on the files ECHOMAIL.BBS and
NETMAIL.BBS for Hudson and/or ECHOMAIL.JAM for JAM areas
to indicate new messages. If these files are found to be
pointing to a message that does not qualify for export,
a complete scan of the message base (or JAM base) will
be performed.

(I hate to quote the docs, but there you go ;)).

Bye, Wilfred.

--- FMail-lnx64 2.1.0.18-B20170815
* Origin: FMail development HQ (2:280/464)

Hi Ozz,

On 2019-02-14 10:14:50, you wrote to me:

Isn't that what the .jlr files in the JAM message base are for? They
contain the last read "pointers" for each user, and they are updated
everytime fmail updates the jam area files, as far as I remember...?

Yes, but hitting 1000+ JLR files all over a drive *is slow*

And would probably less then a second on a modern ssd drive. ;)

versus, SCANTHIS.LST from me to you - you only hit 5 to 10 echos that
have been active since I signaled FMAIL SCAN.

And the options I mentioned in my previous message will do for this?

And vise versa, FMAIL and Rhenium run in a seperate thread/window from
the BBS which is actually running right now behind GameSrv.exe in
dynamic windows. * This will *SERIOUSLY* change when I migrate this to
my Linux rack. However, in either case, I do not "SCAN" after every caller, and I do not "TOSS" after every mailer session.

I think you could. I don't run a bbs, so scan after a caller isn't an issue on my system. But I do run a toss after each mailer session. It's so fast, I don't have to give it another thought:

wilnux5:/home/fido/log # grep 'Toss Active' fmail.log | tail
21:10:08.860 Toss Active: 0.025 sec.
21:12:17.939 Toss Active: 0.080 sec.
21:13:15.650 Toss Active: 0.021 sec.
21:14:39.782 Toss Active: 0.0051 sec.
21:38:05.132 Toss Active: 0.030 sec.
21:39:15.685 Toss Active: 0.030 sec.
21:40:21.420 Toss Active: 0.044 sec.
21:42:20.530 Toss Active: 0.0074 sec.
21:59:15.780 Toss Active: 0.041 sec.
22:03:12.721 Toss Active: 0.036 sec.

I do this, as I get 100's of port scans a minute that would normally trigger a "FMAIL TOSS" and really eat CPU...

I get hardly any port scans on the binkp port. And binkd only triggers a toss if there was a true mailer session, and pkt's received...

Currently I and the single other test node, only use FMail with Binkly
Style Outbound (BSO), and Golded as editor. And no bbs. So it could
use some testing in other environments...

If you do not mind *crazy* ideas to steamline my systems across the US - I will gladly share ideas.

Ideas are ok, but I've very little time for working on fmail. Work, living, etc... ;)


Bye, Wilfred.

--- FMail-lnx64 2.1.0.18-B20170815
* Origin: FMail development HQ (2:280/464)

Hello Wilfred.

14 Feb 19 22:43, you wrote to me:

There's already this option in the config:

Tossed Areas List

Writes a list of echomail areas in which mail has been
tossed to the specified file.

I don't use it myself, but it seems just what you want.

* I didn't see that one, I saw:

There is the "echomail.jam" that's read by fmail scan. Golded for
instance creates it.

Just not sure of the format - is that in the DOCs too???

(I hate to quote the docs, but there you go ;)).

I skimmed the DOCs, and had seen ECHOMAIL.JAM statement - but, assumed that was a RA 2.x feature and didn't want to go back through their STRUCT files to find it... thus, I asked. ;-)

*I* do not mind making my products seamless with other solutions. My mailer has 4 setting you change and d'Bridge becomes the tosser/manager. Since FMail is multi-platform also, I do not mind making the BBS and the MAILER work seamlessly with FMail...


Ozz

--- FMail-W32 2.0.1.4
* Origin: Richmond VA (RVA) Fidonet Support (1:275/362)

Hello Wilfred.

14 Feb 19 22:56, you wrote to me:

wilnux5:/home/fido/log # grep 'Toss Active' fmail.log | tail

C:\BBS\FMAIL>grep "Toss Active" TOSSER.LOG
22:45:11 Toss Active: 0.015 sec.
22:46:00 Toss Active: 0.171 sec.
13:30:16 Toss Active: 2.125 sec.
13:52:33 Toss Active: 0.172 sec.
14:09:49 Toss Active: 0.171 sec.
16:16:57 Toss Active: 0.203 sec.
18:45:29 Toss Active: 0.203 sec.
18:58:05 Toss Active: 0.218 sec.
18:58:19 Toss Active: 0.203 sec.
19:09:40 Toss Active: 7.884 sec.
13:46:53 Toss Active: 0.313 sec.
13:54:08 Toss Active: 4.218 sec.
14:00:51 Toss Active: 0.218 sec.
14:01:46 Toss Active: 1.859 sec.
10:04:48 Toss Active: 1.860 sec.
18:23:26 Toss Active: 1.781 sec.
18:24:20 Toss Active: 0.406 sec.
10:20:22 Toss Active: 1.616 sec.

As you can see, it is not as fast as your Linux system, it is not bad though. The biggest delay is unpacking my daily digest of FSXNET. Those are the times FMail exceed a 1000ms.

I do this, as I get 100's of port scans a minute that would
normally trigger a "FMAIL TOSS" and really eat CPU...

I get hardly any port scans on the binkp port. And binkd only triggers
a toss if there was a true mailer session, and pkt's received...

Correct, however, if I put FMail Scan in my STARTBBS.BAT - I would scan way too often. ;-)

** Off to research ECHOMAIL.JAM....

Ozz

--- FMail-W32 2.0.1.4
* Origin: Richmond VA (RVA) Fidonet Support (1:275/362)

On 15 Feb 2019 at 10:43a, Ozz Nixon pondered and said...

though. The biggest delay is unpacking my daily digest of FSXNET. Those are the times FMail exceed a 1000ms.

That kind of makes me smile :)

Best, Paul

--- E:avon@bbs.nz ------ W:bbs.nz ---
--- K:keybase.io/avon --------------

--- Mystic BBS v1.12 A43 2019/02/15 (Windows/32)
* Origin: Agency BBS | Dunedin, New Zealand | agency.bbs.nz (3:770/100)

Hi Ozz,

On 2019-02-15 10:36:32, you wrote to me:

There is the "echomail.jam" that's read by fmail scan. Golded for
instance creates it.

Just not sure of the format - is that in the DOCs too???

I skimmed the DOCs, and had seen ECHOMAIL.JAM statement - but, assumed that was a RA 2.x feature and didn't want to go back through their
STRUCT files to find it... thus, I asked. ;-)

It's not in the fmail of golded docs. And my guess is, it's a RA thing...


Bye, Wilfred.

--- FMail-lnx64 2.1.0.18-B20170815
* Origin: FMail development HQ (2:280/464)

Hi Ozz,

On 2019-02-15 10:43:23, you wrote to me:

wilnux5:/home/fido/log # grep 'Toss Active' fmail.log | tail

C:\BBS\FMAIL>grep "Toss Active" TOSSER.LOG
22:45:11 Toss Active: 0.015 sec.
22:46:00 Toss Active: 0.171 sec.
13:30:16 Toss Active: 2.125 sec.
13:52:33 Toss Active: 0.172 sec.
14:09:49 Toss Active: 0.171 sec.
16:16:57 Toss Active: 0.203 sec.
18:45:29 Toss Active: 0.203 sec.
18:58:05 Toss Active: 0.218 sec.
18:58:19 Toss Active: 0.203 sec.
19:09:40 Toss Active: 7.884 sec.
13:46:53 Toss Active: 0.313 sec.
13:54:08 Toss Active: 4.218 sec.
14:00:51 Toss Active: 0.218 sec.
14:01:46 Toss Active: 1.859 sec.
10:04:48 Toss Active: 1.860 sec.
18:23:26 Toss Active: 1.781 sec.
18:24:20 Toss Active: 0.406 sec.
10:20:22 Toss Active: 1.616 sec.

As you can see, it is not as fast as your Linux system, it is not bad though. The biggest delay is unpacking my daily digest of FSXNET. Those

are

the times FMail exceed a 1000ms.

This might also be influenced by periodic tossing. In my case when I toss immediately after the mailer session. The received files are almost certainly still in memory caches. When you toss periodic, depending on how busy your system is, and the resources available, that might not be the case...

Bye, Wilfred.

--- FMail-lnx64 2.1.0.18-B20170815
* Origin: FMail development HQ (2:280/464)

Hi! Wilfred,

On 16 Feb 19 12:04, you wrote to Ozz Nixon:

There is the "echomail.jam" that's read by fmail scan. Golded
for instance creates it.

Just not sure of the format - is that in the DOCs too???

I skimmed the DOCs, and had seen ECHOMAIL.JAM statement - but,
assumed that was a RA 2.x feature and didn't want to go back
through their STRUCT files to find it... thus, I asked. ;-)

It's not in the fmail of golded docs. And my guess is, it's a RA
thing...

It's known more widely. From what I've been able to find out in 5 minute's checking distro archives tossers like: CrahMail, FastEcho, FMail 1.22, GEcho, and the TimEd editor were aware of the file(s). Strangely Squish didn't. ;)

Cheers,
Paul.

... Whenever I get a grip on reality, the handle falls off.
--- GoldED+/LNX 1.1.5-b20130515
* Origin: Quinn's Rock - Live from Paul's Xubuntu desktop! (3:640/1384)

On 2019 Feb 16 12:04:48, you wrote to Ozz Nixon:

I skimmed the DOCs, and had seen ECHOMAIL.JAM statement - but,
assumed that was a RA 2.x feature and didn't want to go back through
their STRUCT files to find it... thus, I asked. ;-)

It's not in the fmail of golded docs. And my guess is, it's a RA thing...

no... RA/FD/FE implemented it since they added it as part of a feature request... it was taken up by many others as well... the problem is that there are two or three similar files that provide the information but in slightly different ways... echomail.jam and netmail.jam are for _outbound_ local posts that need to be scanned out of the message base... some try to use this file for inbound which is the exact opposite of what it is intended for...

the other files have another name and may be more generally used for "sole occupant" type setups like single user BBSes or point systems... especially those that use local sysop reader/editor setups like golded, timed, msged and similar... those can use those inbound files to sort those areas to the top of the mail listing if you want to see them first... i don't see them working very well for multi-user setups like a BBS with multiple users... the last read pointers of the users still need to be consulted no matter what mail arrived recently...

on searching for new messages since a user's last visit, it should be easy enough for any JAM capable package to maintain the two lastread pointers for each user... it should also be able to quickly scan through the JLR files picking up and comparing the lastread pointers for each user with the number of messages in the area the JLR file is for... if a system is too slow doing that, they may want to examine their algorithm and/or system setup...

i do recall, on DOS systems, that there was a major slowdown of scanning files which JAM brought to the forefront... the problem was actually in the file system and the way that the OS managed FAT* areas with large numbers of files... folks thought they were putting (eg) 400 message areas in one directory but they didn't realize they were really putting 1600 files in one directory... splitting the areas into directories with less than 255 _files_ (63 JAM areas) per directory sped the processing up by at least a magnitude...

access speed problems may also be seen on networked drive shares... JAM really should be used from local fast HDs, IMHO...

i like the OOP aspect of the pascal JAM library i used in my code... it was fast and did everything i needed... i don't know how non-OOP linear procedural code would work... if it would gather the needed information from the message base files as quickly or if additional requests would need to be made which could slow the scanning process down...

FWIW: when i was using JAM, i was splitting my areas into a structured directory tree something like this...

\FTNs\
\FTNs\Fidonet\
\FTNs\Fidonet\messages\
\FTNs\Fidonet\messages\netmail\
\FTNs\Fidonet\messages\echomail\
\FTNs\Fidonet\messages\echomail\backbone\
\FTNs\Fidonet\messages\echomail\backbone\1\
\FTNs\Fidonet\messages\echomail\backbone\1\10th_amd.j*
\FTNs\Fidonet\messages\echomail\backbone\A\
\FTNs\Fidonet\messages\echomail\backbone\A\Alaska\Chat.j*
\FTNs\Fidonet\messages\echomail\backbone\A\All\Politics.j*
\FTNs\Fidonet\messages\echomail\backbone\A\Allfix\File.j*
\FTNs\Fidonet\messages\echomail\backbone\A\Allfix\Help.j*
[...]
\FTNs\Fidonet\messages\echomail\backbone\B\
\FTNs\Fidonet\messages\echomail\backbone\B\Bash.j*
\FTNs\Fidonet\messages\echomail\backbone\B\Binkd.j*
[...]
\FTNs\Fidonet\messages\echomail\backbone\C\
[...]
\FTNs\Fidonet\files\FDNs\NODELIST\NODELIST.*
\FTNs\Fidonet\files\FDNs\DAILYLIS.T\DAILYLST.*
[...]
\FTNs\Foobar\messages\netmail\
\FTNs\FooBar\messages\echomail\backbone\A[...]
\FTNs\FooBar\messages\echomail\backbone\B[...]
\FTNs\FooBar\messages\echomail\backbone\C[...]
\FTNs\FooBar\messages\echomail\backbone\D[...]

it made things faster and also easier to maintain... autoadded areas were placed into a special directory for them until their record was updated in the tosser configuration program at which time it was moved to the proper directory in the above tree... i used whatever splitter was used between words as a divider... the ALLFIX_FILE and ALLFIX_HELP areas are good examples of that above... same with gated news groups where you use the dots between the portions of the group name as the directory splitter...

eg: alt.bbs.ads -> [...]\alt\bbs\ads
alt.comp.anti.virus -> [...]\alt.comp.anti.virus

)\/(ark

Always Mount a Scratch Monkey
Do you manage your own servers? If you are not running an IDS/IPS yer doin' it wrong...
... You say I'm a bastard like it's a bad thing.
---
* Origin: (1:3634/12.73)

On 2019-02-16 19:24:56 +0000, mark lewis -> Wilfred van Velzen said:

i do recall, on DOS systems, that there was a major slowdown of
scanning files which JAM brought to the forefront... the problem was
actually in the file system and the way that the OS managed FAT* areas
with large numbers of files... folks thought they were putting (eg) 400 message areas in one directory but they didn't realize they were really putting 1600 files in one directory... splitting the areas into
directories with less than 255 _files_ (63 JAM areas) per directory
sped the processing up by at least a magnitude...

Luckily those limitations no longer exist :-)

--
+++ Are we having fun yet? I am!
ATZ|ATDT911|1,2~555

--- FMail-W32 2.0.1.4
* Origin: ExchangeBBS WHQ (1:275/362.0)