Hi!

In case of subject longer then allowed 71 + NUL bytes HPT send the whole
PKT with the incorrect message to bads.

Would it be more practical to be more error tolerant and just cut it to
the allowed limit, or it's considered as modification of messages and
forbidden by some document?

I'm wondering since fidogate has different logic and may be I'll have to
fix it, even if I personally prefer to receive shorten subject than no
message at all.
--- Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
* Origin: Somewhere in the North (2:4500/1.59)

Hello Zhenja,

Friday January 10 2020, Zhenja Kaliuta wrote to All:

In case of subject longer then allowed 71 + NUL bytes HPT send the
whole PKT with the incorrect message to bads.

Would it be more practical to be more error tolerant and just cut it
to the allowed limit, or it's considered as modification of messages
and forbidden by some document?

Fidonet Policy v.4.07

========= Here the quote begins ===========
2.1.5 No Alteration of Routed Mail

You may not modify, other than as required for routing or other technical purposes, any message, netmail or echomail, passing through the system from
one FidoNet node to another.
========= Here it ends =============

Michael

... node (at) f1042 (dot) ru
--- GoldED+/LNX 1.1.5-b20170303
* Origin: Moscow, Russia (2:5020/1042)

Hi, Michael!

On Fri, 10 Jan 2020 21:40:01 +0200 Michael Dukelsky writes:

In case of subject longer then allowed 71 + NUL bytes HPT send the
whole PKT with the incorrect message to bads.

Would it be more practical to be more error tolerant and just cut
it to the allowed limit, or it's considered as modification of
messages and forbidden by some document?

Fidonet Policy v.4.07

========= Here the quote begins ===========
2.1.5 No Alteration of Routed Mail

You may not modify, other than as required for routing or other technical purposes, any message, netmail or echomail, passing through the system

from

one FidoNet node to another.
========= Here it ends =============

Isn't it an "other technical purpose"? ;)


--- Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
* Origin: Somewhere in the North (2:4500/1.59)

Re: hpt long subjects and bad packets
By: Zhenja Kaliuta to All on Fri Jan 10 2020 17:37:27

In case of subject longer then allowed 71 + NUL bytes HPT
send the whole PKT with the incorrect message to bads.

what FTN software is creating and/or passing on messages with subject lines longer than the spec for packed messages allows??? whichever software that is needs to be fixed to stop the defective pkts from being generated and sent out in the first place...



)\/(ark
--- SBBSecho 3.10-Linux
* Origin: SouthEast Star Mail HUB - SESTAR (1:3634/12)

Hello Zhenja!

10 Jan 20, Zhenja Kaliuta wrote to Michael Dukelsky:

In case of subject longer then allowed 71 + NUL bytes HPT send
the whole PKT with the incorrect message to bads.

Would it be more practical to be more error tolerant and just
cut it to the allowed limit, or it's considered as modification
of messages and forbidden by some document?

No. If you work around the bugs of others you will build a pile of shards.

2.1.5 No Alteration of Routed Mail
You may not modify, other than as required for routing or other
technical purposes, any message, netmail or echomail, passing

Isn't it an "other technical purpose"? ;)

Work around a "standard violation" should never become a technical purpose.

Fix the source of the problem. If it can't be fixed don't allow the broken to enter. That's straight, clear and simple.

Regards

Kai

--- GoldED+/LNX 1.1.4.7
* Origin: Monobox (2:240/77)

10 Jan 20 21:48, Zhenja Kaliuta wrote to Michael Dukelsky:

In case of subject longer then allowed 71 + NUL bytes HPT send the
whole PKT with the incorrect message to bads.

Would it be more practical to be more error tolerant and just cut
it to the allowed limit, or it's considered as modification of
messages and forbidden by some document?

Fidonet Policy v.4.07

========= Here the quote begins ===========
2.1.5 No Alteration of Routed Mail

You may not modify, other than as required for routing or other technical
purposes, any message, netmail or echomail, passing through the system
from one FidoNet node to another.
========= Here it ends =============

Isn't it an "other technical purpose"? ;)

It depends who you ask. ;)

I made a quick test: I set my terminal to 132x45 and entered a message to JAM base with GoldED with subject of 100 chars. It stayed there until "hpt scan" when it was cut to 72 chars. However this happened in my msgbase, not in transit mail.

So it would be best to fix fidogate (?) not to send out illegal pkt's.

'Tommi

---
* Origin: 2001:470:1f15:cb0:f1d0:2:221:1 (2:221/1)

Hi, mark!

On Fri, 10 Jan 2020 19:51:55 +0200 mark lewis writes:

In case of subject longer then allowed 71 + NUL bytes HPT
send the whole PKT with the incorrect message to bads.

what FTN software is creating and/or passing on messages with
subject lines longer than the spec for packed messages allows???
whichever software that is needs to be fixed to stop the defective
pkts from being generated and sent out in the first place...

No doubts. No arguments against that.

But software may have bugs and it's sad to lose otherwise valid data
because of that.

But I understand the general attitude, thanks you for the input.

BTW, in RFC world it's pretty different. Ex.: rfc2045

```
(5) Encoded lines must not be longer than 76 characters,
not counting the trailing CRLF. If longer lines are
found in incoming, encoded data, a robust
implementation might nevertheless decode the lines, and
might report the erroneous encoding to the user.
```
which looks more human friendly for me.

And the original question can be splitted basically into 2:

- is it ok to handle the situation without failing the pkt?
- is it ok to make the pkt standard complied?


I got your answers, thanks a lot!


--- Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
* Origin: Somewhere in the North (2:4500/1.59)

Hi, Kai!

On Sat, 11 Jan 2020 07:23:10 +0200 Kai Richter writes:

In case of subject longer then allowed 71 + NUL bytes HPT send
the whole PKT with the incorrect message to bads.

Would it be more practical to be more error tolerant and just
cut it to the allowed limit, or it's considered as modification
of messages and forbidden by some document?

No. If you work around the bugs of others you will build a pile of
shards.

Well, in general in the world there are other opinions:

https://tools.ietf.org/html/rfc1122

```
1.2.2 Robustness Principle

[...]

"Be liberal in what you accept, and
conservative in what you send"
```

https://tools.ietf.org/html/rfc7103

```Advice for Safe Handling of Malformed Messages```

2.1.5 No Alteration of Routed Mail
You may not modify, other than as required for routing or other
technical purposes, any message, netmail or echomail, passing

Isn't it an "other technical purpose"? ;)

Work around a "standard violation" should never become a technical purpose.

Valid point, thanks.

Fix the source of the problem. If it can't be fixed don't allow the
broken to enter. That's straight, clear and simple.

--- Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
* Origin: Somewhere in the North (2:4500/1.59)

Hi, Tommi!

On Sat, 11 Jan 2020 09:53:00 +0200 Tommi Koivula writes:

In case of subject longer then allowed 71 + NUL bytes HPT send
the whole PKT with the incorrect message to bads.

Would it be more practical to be more error tolerant and just
cut it to the allowed limit, or it's considered as modification
of messages and forbidden by some document?

Fidonet Policy v.4.07

========= Here the quote begins ===========
2.1.5 No Alteration of Routed Mail

You may not modify, other than as required for routing or other technical
purposes, any message, netmail or echomail, passing through the system
from one FidoNet node to another.
========= Here it ends =============

Isn't it an "other technical purpose"? ;)

It depends who you ask. ;)

I made a quick test: I set my terminal to 132x45 and entered a
message to JAM base with GoldED with subject of 100 chars. It
stayed there until "hpt scan" when it was cut to 72 chars. However
this happened in my msgbase, not in transit mail.

Agree, it's a different case.

So it would be best to fix fidogate (?) not to send out illegal
pkt's.

Oh, just to avoid wrong conclusions -- the original question was not
because of fidogate producing incorrect packets, in this place it
behaves correctly.

I mentioned it in connection with handling incoming packets. I compared
it to what hpt is doing (found one problem actually) and started
thinking of what would be the best way.


--- Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
* Origin: Somewhere in the North (2:4500/1.59)

Hi Michael!

Friday January 10 2020 you wrote to Zhenja Kaliuta the following:

As usual, FPD contradicts with itself. :)

2.1.5 No Alteration of Routed Mail

You may not modify ... echomail

Echomail is not routed nowadays. It is a sort of _direct_ mail from echo uplink to echo downlink. :)

Yours
Andrew Kolchoogin.

... Those who sacrifices freedom for security deserves neither
--- Pen on parchment
* Origin: -= Castle of Silence =- (2:5020/290.22)

Re: Re: hpt long subjects and bad packets
By: Zhenja Kaliuta to Mark lewis on Sat Jan 11 2020 09:49:17

- is it ok to handle the situation without failing the pkt?
- is it ok to make the pkt standard complied?

in my personal and professional opinion, the answer to both questions is a resounding "yes" but with at least the following caveat...

1. HPT is the first FTN processor to handle the defective pkt
after the flawed (gating?) software has created it...

once the defective pkts are already in the distribution stream, meaning another FTN tosser has processed it, then no... mark them as bad and set them aside with a proper logged reason...

eg: foobar.pkt - packed msg #XXX - subject field too long

or some such... possibly also record the identity of the processor that created the defective packet if that's not already being done... that info can be gathered from the packet header... one might also want to log the above when the processor repairs such a defective message and log an additional message stating the repair decision... logging bad pkts is generally already done and adding logging that the repair of message #xxx is being done is trivial...

others may or may not feel the same way...

i fully agree that this is a technical problem which is allowed for by current fidonet policy but also keep the above caveat in mind... other FTNs may have different policy, though, and that should also be considered... it may lead to the "repairing tosser" to perform this fix or sidelining based on the FTN the packets belong to which would basically mean a setting per FTN... possibly something like

FTN zone(s): 1-4095
FTN domain: xxxxxxxx
Repair defective pkts: yes/no

in the configuration settings... i'm thinking of this in this manner because if the pkt is only 3D/4D, then the domain info is not available and the decision would be made based on the zone number... if the pkt is 5D, then the domain info is available and the decision would be made on that... YMMV is applicable here, though...


)\/(ark
--- SBBSecho 3.10-Linux
* Origin: SouthEast Star Mail HUB - SESTAR (1:3634/12)

Re: hpt long subjects and bad packets
By: Kai Richter to Zhenja Kaliuta on Sat Jan 11 2020 07:23:10

Isn't it an "other technical purpose"? ;)

Work around a "standard violation" should never become a technical

purpose.

Fix the source of the problem. If it can't be fixed don't allow the broken

to enter. That's straight, clear and simple.

keeping in mind my previous message on this, i also highly agree with this and still look to the caveat i listed in that previous message...

ew (TINW) really need to know the name of this defective software creating pkts with too long subject lines... it is possible there's a newer/older version that doesn't have this defect and the originating system can be pointed to this version to replace their defective one...


)\/(ark
--- SBBSecho 3.10-Linux
* Origin: SouthEast Star Mail HUB - SESTAR (1:3634/12)

Re: hpt long subjects and bad packets
By: Andrew Kolchoogin to Michael Dukelsky on Sat Jan 11 2020 10:47:44

As usual, FPD contradicts with itself. :)

there is a huge difference between technical specification and network policy... what was quoted was policy... FTS-0001 is a little confusing in its description of field formats in some cases but section A4 seems to be missed/ignored/skipped over by a lot of people...

----->8 snip 8<-----

A. Introduction

[...]

4. Data Description

A language specific notation was avoided. Please help stamp out
environmental dependencies. Only you can prevent PClone market
dominance. Don't panic, there are rectangular record layouts too.

(* non-terminals *)
UpperCaseName - to be defined further on

(* literals *)
"ABC" - ASCII character string, no termination implied
nnH - byte in hexadecimal

(* terminals *)
someName - 16-bit integer, low order byte first (8080 style)
someName[n] - field of n bytes
someName[.n] - field of n bits

someName(n) - Null terminated string allocated n chars (incl Null)
someName{max} - Null terminated string of up to max chars (incl Null)

(* punctuation *)
a b - one 'a' followed by one 'b'
( a | b ) - either 'a' or 'b', but not both
{ a } - zero or more 'a's
[ b ] - zero or one 'b'
(* comment *) - ignored

(* predeclared constant *)
Null = 00H

----->8 snip 8<-----

so, if the subject field of a packed message is defined as

+-----------------------+-----------------------+
| subject |
~ max 72 bytes ~
| null terminated |
+-----------------------+-----------------------+

that means 71 bytes plus the terminating 0x00 byte...

a lot of us know this but hobby coders may, as mentioned above, miss/ignore/skip section A4 and cause a defect in their code... the rest of us can either work around it as many have done over the years or we can reject those defects and bring the problem to the coder's attention...


)\/(ark
--- SBBSecho 3.10-Linux
* Origin: SouthEast Star Mail HUB - SESTAR (1:3634/12)

Hi, mark!

On Sat, 11 Jan 2020 08:16:43 +0200 mark lewis writes:

Isn't it an "other technical purpose"? ;)

Work around a "standard violation" should never become a technical
purpose.

Fix the source of the problem. If it can't be fixed don't allow
the broken to enter. That's straight, clear and simple.

keeping in mind my previous message on this, i also highly agree
with this and still look to the caveat i listed in that previous message...

Thanks for the detailed reply.

ew (TINW) really need to know the name of this defective software
creating pkts with too long subject lines... it is possible there's
a newer/older version that doesn't have this defect and the
originating system can be pointed to this version to replace their defective one...

There is no problem there (anymore). It was some newly written software
and the author is aware of it and has fixed it AFAIU.



--- Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
* Origin: Somewhere in the North (2:4500/1.59)

Fix the source of the problem. If it can't be fixed don't allow the
broken to enter. That's straight, clear and simple.

Maybe we should block all PKTs generated by Mystic BBS then? And also PKTs from hpt itself? (which doesn't set the JAM OADDRESS and rewrites the message date for Squish messages.



* Origin: kakistocracy (2:280/464.47)
--- Platinum Xpress/Win/WINServer v7.0

Hello Zhenja!

11 Jan 20, Zhenja Kaliuta wrote to Mark lewis:

But software may have bugs and it's sad to lose otherwise valid data because of that.

I don't want to accept that argument because there is no clear line what is "a goog bug" that needs to be corrected by others and "a bad bug" that causes more harm than being useful. And even "good bugs" would create a bunch of workarounds with the risk of interferring each other by time.

It's like a water bucket with a small hole. You can workaround by adding water up to full and run faster to your destination. Or you can place your hand on the hole. With more holes you will have less water at the destination or you need more hands to cover the holes. What do you do if there is no water in the bucket when you arrived at the destination? Throw the bucket away and get a new one? Or at last start fixing the holes?

BTW, in RFC world it's pretty different. Ex.: rfc2045

If there is a better bucket throw away the old one and use the new one.

Regards

Kai

--- GoldED+/LNX 1.1.4.7
* Origin: Monobox (2:240/77)

Hello Zhenja!

11 Jan 20, Zhenja Kaliuta wrote to Kai Richter:

Well, in general in the world there are other opinions:

https://tools.ietf.org/html/rfc1122

Other opinions of another world.

I do understand why people would like to "improve" fidonet or "advance" FTS. But hey, that job is already done. We already got rid of the numeric address system and use letters and numbers together, we can display pictures and videos next to text, we reduced the transfer time and many new standards where invented, even the name was refreshed to "The internet".

Regards

Kai

--- GoldED+/LNX 1.1.4.7
* Origin: Monobox (2:240/77)

Hello mark!

11 Jan 20, mark lewis wrote to Zhenja Kaliuta:

- is it ok to handle the situation without failing the pkt?
- is it ok to make the pkt standard complied?

in my personal and professional opinion, the answer to both questions
is a resounding "yes" but with at least the following caveat...

Sorry, i didn't noticed your next mail where you highly agreed with me before writing. Please ignore this mail. I send it because i added some details of the consequences how works arounds can block each other.

1. HPT is the first FTN processor to handle the defective pkt
after the flawed (gating?) software has created it...

The first FTN processor is the flawed software that generates non FTS pkts.
No matter what that software normally does - it has a module that exports FTN pkt and that job should be done in accordance to FTS.

once the defective pkts are already in the distribution stream,
meaning another FTN tosser has processed it, then no... mark them as
bad and set them aside with a proper logged reason...

The result is that only direct link systems are flooded with broken content. That could be a nice idea, but because of the actual practice for network stability improvement called multi-link routing, there are many "first HPT tosser" that would receive the broken pkt.

This is a good example that work arounds do interfere with other work arounds. Your idea would work for a straight star network but not for a mesh type network.

or some such... possibly also record the identity of the processor
that created the defective packet if that's not already being done...
one might also want to log the above when the processor repairs such
a defective message and log an additional message stating the repair decision... logging bad pkts is generally already done and adding
logging that the repair of message #xxx is being done is trivial...

others may or may not feel the same way...

Instead of all other network members waste their resources for logging, fixing and accepting broken subjects within the network only one system needs to find a better solution how to transfer too long subject lines into the standard pkt format. A short and simple way is to copy the full subject line into the message text and cut the subject short plus adding a timestamp. The timestamp is required if the subject line is edited in the too long part only. This change must be visible in the short line somehow to keep reply linking searching and sorting by subject line working as usual.

i fully agree that this is a technical problem which is allowed for
by current fidonet policy

I don't think so. I'm not in details but it doesn't make sense. The FTS are the standard valid for any FTS based networks to keep the networks running without problems. The FTS sets the frame. If the subject line is limited any FTN software have to be compliance to that limit - or it is not a FTS software.

The purpose of the "other technical problems" in the policy can't be used to transform the network into a trashcan and leave the responsibility to find useful stuff in the trash to us.

The problem of non FTS conform subject lines is a problem out of the FTN/FTS responsibility because it's not a valid FTS pkt. Period. The decision of HPT to move that non standard content to bad is correct. The decision not to modifiy and not to route the broken content is the only way to tell the source of the problem that his software has a serious bug that needs to be fixed.

basically mean a setting per FTN... possibly something like
Repair defective pkts: yes/no

That would make it more complex. Keep it simple. It's not your/our job to fix incoming trash. HPT should detect a broken pkt, log it and move it to bad. That's how the broken pkt was found so hpt already works as designed.

Regards

Kai

--- GoldED+/LNX 1.1.4.7
* Origin: Monobox (2:240/77)

Hello mark!

11 Jan 20, mark lewis wrote to Kai Richter:

ew (TINW) really need to know the name of this defective software
creating pkts with too long subject lines... it is possible there's a newer/older version that doesn't have this defect and the originating system can be pointed to this version to replace their defective
one...

Good point. Additionally we all know about the development status of very old FTN software. If that software can't be fixed then there may be an alternate software that does the same job.

Regards

Kai

--- GoldED+/LNX 1.1.4.7
* Origin: Monobox (2:240/77)

Hello Zhenja!

11 Jan 20, Zhenja Kaliuta wrote to Mark lewis:

There is no problem there (anymore). It was some newly written
software and the author is aware of it and has fixed it AFAIU.

That's a positiv surprise. Wonders will never cease. ;-)

Regards

Kai

--- GoldED+/LNX 1.1.4.7
* Origin: Monobox (2:240/77)

Hello Oli!

11 Jan 20, Oli wrote to Kai Richter:

Fix the source of the problem. If it can't be fixed don't allow
the broken to enter. That's straight, clear and simple.

Maybe we should block all PKTs generated by Mystic BBS then?

Well, i'm emotionally affected so i'm the wrong person for judgement. If you would start a seriously vote i would mark the YES field. But thats because some mails from that software crash my terminal output to unreadable. I have to quit my editor then, do a terminal reset and start again carefully skipping the broken mail. That is very annoying sometimes.

And also PKTs from hpt itself? (which doesn't set the JAM OADDRESS and rewrites the message date for Squish messages.

Actually i have no clue if msgbase formats are part of the FTS. I never noticed the msgbase format of incoming mails. I can't see if you use JAM or squish. Anyway your mail is stored in squish on my system. That's why i can't see a reason to drop a bug report.

Regards

Kai

--- GoldED+/LNX 1.1.4.7
* Origin: Monobox (2:240/77)

13 Jan 20 12:27, you wrote to me:

Fix the source of the problem. If it can't be fixed don't allow
the broken to enter. That's straight, clear and simple.

Maybe we should block all PKTs generated by Mystic BBS then?

Well, i'm emotionally affected so i'm the wrong person for judgement.
If you would start a seriously vote i would mark the YES field. But
thats because some mails from that software crash my terminal output
to unreadable. I have to quit my editor then, do a terminal reset and start again carefully skipping the broken mail. That is very annoying sometimes.

I'm emotionally affected too. Most of the time I'm reading fido mails on my 4.8" phone (ssh session to my headless Raspberry). The screen is a bit small to display 80 columns in a comfortable font size. Mystic's hard wrapping of all mail that passes through to 80 chars/line is very annoying.

And also PKTs from hpt itself? (which doesn't set the JAM OADDRESS
and rewrites the message date for Squish messages.

Actually i have no clue if msgbase formats are part of the FTS. I
never noticed the msgbase format of incoming mails. I can't see if you
use JAM or squish. Anyway your mail is stored in squish on my system. That's why i can't see a reason to drop a bug report.

As far as I understand it from the documentation hpt always does import/export in one pass, is that correct? But What about echoareas that get rescanned? Around 50% of the mails (or a little bit less) will have a modified timestamp (DOS time format with 2 second granularity). To be clear, Squish message base format has a field for storing the original FTN time field, it is just not used by hpt (if I understand the C sources correctly).


* Origin: kakistocracy (2:280/464.47)
--- Platinum Xpress/Win/WINServer v7.0

Hello Oli,

Tuesday January 14 2020, Oli wrote to Kai Richter:

And also PKTs from hpt itself? (which doesn't set the JAM
OADDRESS and rewrites the message date for Squish messages.

As far as I understand it from the documentation hpt always does import/export in one pass, is that correct? But What about echoareas
that get rescanned? Around 50% of the mails (or a little bit less)
will have a modified timestamp (DOS time format with 2 second granularity). To be clear, Squish message base format has a field for storing the original FTN time field, it is just not used by hpt (if I understand the C sources correctly).

Could you please send me a bug report with full description of actions to reproduce the error with an example of the message base and pkt taking part in the actions. I am very busy at the moment but I'll try to investigate it when time permits.

Michael

... node (at) f1042 (dot) ru
--- GoldED+/LNX 1.1.5-b20170303
* Origin: Moscow, Russia (2:5020/1042)

And also PKTs from hpt itself? (which doesn't set the
JAM OADDRESS and rewrites the message date for Squish
messages.

As far as I understand it from the documentation hpt
always does import/export in one pass, is that correct?
But What about echoareas that get rescanned? Around 50%
of the mails (or a little bit less) will have a modified
timestamp (DOS time format with 2 second granularity). To
be clear, Squish message base format has a field for
storing the original FTN time field, it is just not used
by hpt (if I understand the C sources correctly).

Could you please send me a bug report with full
description of actions to reproduce the error with an
example of the message base and pkt taking part in the
actions. I am very busy at the moment but I'll try to
investigate it when time permits.

I'm not using hpt and I cannot provide example files, but the problem is simple:

a squish tosser should write the date string to the __ftsc_date field when importing mails and should use the __ftsc_date field (if it is not empty) on export. see the squish spec. smapi seems to support that field, but hpt doesn't use it.

the JAM problem:
it was mentioned in some other echoarea that hpt doesn't set the OADDRESS for echomail and that the origin address is missing on rescans. i also couldn't find a place in the sources where the OADDRESS is set.

jammnntpd even has a workaround for missing OADDRESS fields.

please correct me if i'm wrong. if i had the time, i would do some testing with hpt.

and thank you for your hpt support and bug fixing, michael.



* Origin: kakistocracy (2:280/464.47)
--- Platinum Xpress/Win/WINServer v7.0
6

Hello Oli,

Tuesday January 14 2020, Oli wrote to Michael Dukelsky:

a squish tosser should write the date string to the __ftsc_date field
when importing mails and should use the __ftsc_date field (if it is
not empty) on export. see the squish spec. smapi seems to support that field, but hpt doesn't use it.

the JAM problem:
it was mentioned in some other echoarea that hpt doesn't set the
OADDRESS for echomail and that the origin address is missing on
rescans. i also couldn't find a place in the sources where the
OADDRESS is set.

jammnntpd even has a workaround for missing OADDRESS fields.

OK, thank you, I'll look at it when I have time.

Michael

... node (at) f1042 (dot) ru
--- GoldED+/LNX 1.1.5-b20170303
* Origin: Moscow, Russia (2:5020/1042)

14 Jan 20 15:12, Oli wrote to Michael Dukelsky:

I'm not using hpt and I cannot provide example files, but the problem is simple:

a squish tosser should write the date string to the __ftsc_date field when importing mails and should use the __ftsc_date field (if it is not empty)

on

export. see the squish spec. smapi seems to support that field, but hpt

doesn't

use it.

Which field is that? Can I see it in GoldED hexdump pressing "i" ?

These date's are found in your message in my squish base, tossed by hpt.

=== Cut ===
DateTime : 14 Jan 20 15:12:47
OrigAddr : 2:280/464.47
DestAddr : 2:221/1.0
See : 92, 0, 0, 0, 0, 0, 0, 0, 0
Attr : 00020000h (00000000000000100000000000000000b)
DateWritten : 2020-01-14 15:12:46 (7997502Eh)
DateArrived : 2020-01-14 16:13:52 (81BA502Eh)
UTC-Offset : 0
=== Cut ===

'Tommi

---
* Origin: 2001:470:1f15:cb0:f1d0:2:221:1 (2:221/1)

a squish tosser should write the date string to the
__ftsc_date field when importing mails and should use the
__ftsc_date field (if it is not empty) on export. see the
squish spec. smapi seems to support that field, but hpt
doesn't use it.

Which field is that? Can I see it in GoldED hexdump
pressing "i" ?

Good question. I only discovered "i" recently by hitting the wrong key and don't know much about it ...

These date's are found in your message in my squish base,
tossed by hpt.

... but DateTime looks exactly like the value __ftsc_date should have. So I guess I was at least partly wrong. hpt does write __ftsc_date.

=== Cut ===
DateTime : 14 Jan 20 15:12:47
OrigAddr : 2:280/464.47
DestAddr : 2:221/1.0
See : 92, 0, 0, 0, 0, 0, 0, 0, 0
Attr : 00020000h
(00000000000000100000000000000000b)
DateWritten : 2020-01-14 15:12:46 (7997502Eh)
DateArrived : 2020-01-14 16:13:52 (81BA502Eh)
UTC-Offset : 0
=== Cut ===

* Origin: kakistocracy (2:280/464.47)
--- Platinum Xpress/Win/WINServer v7.0

Hello Oli!

14 Jan 20, Oli wrote to Kai Richter:

if you use JAM or squish. Anyway your mail is stored in squish on
my system. That's why i can't see a reason to drop a bug report.

As far as I understand it from the documentation hpt always does import/export in one pass, is that correct?

I don't know. There is control for import by "hpt toss" and control for export via "hpt scan". But for routing i don't know if it's possible to import pkt to local msgbase without exporting the mail to all linked systems. That option doesn't make sense to me. If the user deletes mails before they are forwarded to the linked systems that would be censorship and against the "no modification of in transit mail" of the policy.

But i found another function that reveals that the date field is common for modification:

***
@item Syntax:
@code{processPkt <string>}
@item Example:
@code{processPkt pktdate}

Space char and name of the current file "*.pkt" will be append into end of <string> and <string> willl be executed before tossing each pkt file. You
are able to fix your pkts using pktdate or any other tool before tossing
them. Note that pkt file may be renamed depending of @code{tossingExt}
token value.
***

In the past when i used OS/2 there was a tool called pktsort that could be hook at the same place to sort the mails by date. That fixed the situation that the answer was visible before the question in the msgbase.

For hpt scan there is an option to bypass netmail:

'When PackNetMailOnScan is "on" (default) hpt packs netmail when doing
"hpt scan" and netmail area is found in EchoTossLog file. When it is
"off" hpt leaves netmail area(s) in EchoTossLog file until "hpt pack" is invoked.'

But What about echoareas that get rescanned? Around 50% of the mails
(or a little bit less) will have a modified timestamp (DOS time format
with 2 second granularity).

That "adjusted" timestamp is common to me since i use fidonet. I don't know the decision for that, maybe it's due to 8-bit limitation of the date format on early fidonet computers back in the 80's?

Regards

Kai

--- GoldED+/LNX 1.1.4.7
* Origin: Monobox (2:240/77)