I had noticed that some Unicode characters were not displayed (showing the "rhombus with question mark" instead), precissely those that have a 0x8D byte when encoded in UTF-8. Examples: the cyrillic e (), 0xD1 0x8D; the "thumbs up" emoji (), 0xF0 0x9F 0x91 0x8D.
I've found that this issue can be fixed by adding "-keepsoftcr" to the corresponding line in jamnntpd.xlat, just like with CP866:
================================
read UTF-8 utf-8 -keepsoftcr
================================
I had noticed that some Unicode characters were not displayed (showingthe "rhombus with question mark" instead), precissely those that have a 0x8D byte when encoded in UTF-8. Examples: the cyrillic e (�), 0xD1 0x8D; the "thumbs up" emoji (�), 0xF0 0x9F 0x91 0x8D.
�
I've found that this issue can be fixed by adding "-keepsoftcr" to thecorresponding line in jamnntpd.xlat, just like with CP866:
================================
read UTF-8 utf-8 -keepsoftcr
================================
Done. :)
>> ================================
>> read UTF-8 utf-8 -keepsoftcr
>> ================================
TK> Done. :)
Your message arrived here without the 0x8D chars... :-m
Maybe your tosser removed them?
Your message arrived here without the 0x8D chars... :-m
Maybe your tosser removed them?
Test from HPT. ()
Test from HPT. (👍)
TK> Test from HPT. (👍)
Looks fine now 😍 (U+1F60D, 0xF0 0x9F 0x98 0x8D)
Good! I had stripping soft-cr's on in GEcho, now off. Let's see
how this goes out. (👍)(👍)(👍)(👍)(👍)(👍)(👍)(👍)
Tommi wrote (2023-03-05):
On 5 Mar 2023 16.17, Carlos Navarro wrote:
Test from HPT. (👍)
Looks fine now 😍 (U+1F60D, 0xF0 0x9F 0x98 0x8D)
Good!Does it work?
I had stripping soft-cr's on in GEcho, now off. Let's see how this goes
out.
(👍)(👍)(👍)(👍)(👍)(👍)(👍)(👍)
Did I configure it correctly?
---
* Origin: This site requires JavaScript (2:280/464.47)
I had noticed that some Unicode characters were not displayed (showing the "rhombus with question mark" instead), precissely those that have
a 0x8D byte when encoded in UTF-8. Examples: the cyrillic e (), 0xD1 0x8D; the "thumbs up" emoji (Fl), 0xF0 0x9F 0x91 0x8D.
I've found that this issue can be fixed by adding "-keepsoftcr" to the corresponding line in jamnntpd.xlat, just like with CP866:
================================
read UTF-8 utf-8 -keepsoftcr
================================
Carlos
better is to setup thunderbird to NOT send utf-8 at all
I've found that this issue can be fixed by adding "-keepsoftcr" to the
corresponding line in jamnntpd.xlat, just like with CP866:
================================
read UTF-8 utf-8 -keepsoftcr
================================
problem with jamnntpd is that it does not support multibyte, so only can change singlebyte in diffrent charsets
08/02/2024 6:08, Benny Pedersen -> Carlos Navarro:
I don't understand what you mean. JamNNTPd seems to work fine with UTF-8, except for some issues with quotes.
On Tue, 20 Feb 2024 23:56:32 +0100, Carlos Navarro -> Benny Pedersen wrote:
08/02/2024 6:08, Benny Pedersen -> Carlos Navarro:
I don't understand what you mean. JamNNTPd seems to work fine
with UTF-8, except for some issues with quotes.
Bringing the thread from the Golded echo about Jamnntpd over here.
Here's a test to see if those characters are before level 2 and 3
quotes on this reply.
I've found that this issue can be fixed by adding
"-keepsoftcr" to the corresponding line in jamnntpd.xlat,
just like with CP866:
================================
read UTF-8 utf-8 -keepsoftcr
================================
problem with jamnntpd is that it does not support multibyte,
so only can change singlebyte in diffrent charsets
Bringing the thread from the Golded echo about Jamnntpd over here.
Here's a test to see if those characters are before level 2 and 3
quotes on this reply.
Quoted the whole thing in Golded. I don't see anything here. I also hex dumped the message before replying to it, and didn't see anything in
there either. *shrug*
Quoted the whole thing in Golded. I don't see anything here. I also hex dumped the message before replying to it, and didn't see anything in
there either. *shrug*
If you don't see the crap, it is probably because of
your conversion settings in Golded.
If you don't see the crap, it is probably because of
your conversion settings in Golded.
Or mine. I don't know. :)
On 23 Feb 2024, Tommi Koivula said the following...
If you don't see the crap, it is probably because of
your conversion settings in Golded.
Or mine. I don't know. :)
When viewing this message in Golded it had as empty tear line & the
origin line has a line break. When viewed in Mystic is has a bunch of UTF-8 characters:
https://postimg.cc/gallery/0khhW7t
I don't understand what you mean. JamNNTPd seems to work fine with
UTF-8, except for some issues with quotes.
Bringing the thread from the Golded echo about Jamnntpd over here. Here's a
test to see if those characters are before level 2 and 3 quotes on this reply.
I don't understand what you mean. JamNNTPd seems to work fine
with UTF-8, except for some issues with quotes.
Bringing the thread from the Golded echo about Jamnntpd over here.
Here's a test to see if those characters are before level 2 and 3
quotes on this reply.
Quoted the whole thing in Golded. I don't see anything here. I also hex dumped the message before replying to it, and didn't see anything in there either. *shrug*
When viewing this message in Golded it had as empty tear line & the origin line has a line break. When viewed in Mystic is has a bunch of UTF-8 characters:
https://postimg.cc/gallery/0khhW7t
They are in your message! And I see them:
0160 74 2C 0D 20 C2 A0 43 4E 3E 3E 3E 3E 20 6A 75 73
But I do see crap there. And as you may see, it is not only the quotes, also multiple spaces are converted to crap.
If you don't see the crap, it is probably because of your conversion settings in Golded.
How did my original reply to Carlos look on your end?
I think this is the message you're referring to: https://postimg.cc/gallery/kq34nZw
How did my original reply to Carlos look on your end?
I think this is the message you're referring to: https://postimg.cc/gallery/kq34nZw
They are in your message! And I see them:
0160 74 2C 0D 20 C2 A0 43 4E 3E 3E 3E 3E 20 6A 75 73
And yet they didn't show in your reply to me, like some of the other ones did. This or the last message you wrote, unless you removed them yourself.
I think this is the message you're referring to:
https://postimg.cc/gallery/kq34nZw
Yes, that's the one.. along with most of the others I wrote from Thunderbird. It seems Thunderbird adds non-breaking spaces (or   in html) anywhere there's two or more concurrent spaces next to each other.
I've completely disabled html in TB now, and am forcing plain-text. So we will see in the near future if there's still an issue.
^^
Still there...
^^
Still there...
^^
Still there...
... "Take my advice, I don't use it anyway."
--- Betterbird (Windows)
There is another thunderbird clone called Epyrus. This one allows
posting other charsets than utf-8.
http://www.epyrus.org/
^^
Still there...
Last attempt for the evening, I suppose. This one I changed the original message body to plain text before replying to it.
Frustrating, because I can't check it before the message is sent. It's something Thunderbird is doing after the message is saved.
^^
Still there...
Maybe? Maybe not?
There is another thunderbird clone called Epyrus. This one allows
posting other charsets than utf-8.
Seamonkey also allows posting other than utf-8.
However, the problem is not utf-8, I think it has proved that it is Thunderbird.
Check the .pkt file...
problem with jamnntpd is that it does not support multibyte,
You're right about JamNNTPd not supporting multibyte. It wraps lines at 79 bytes, not at 79 chars. Therefore those that have UTF-8 chars (2 or more bytes) become shorter when wrapped. The more UTF-8 chars a line has, the shorter.
This is noticeable in readers that don't support flowed text (most NNTP clients).
ASCII:
aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou
UTF-8:
áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú
áéíóú áéíóú
problem with jamnntpd is that it does not support multibyte,
You're right about JamNNTPd not supporting multibyte. It wraps lines at
79 bytes, not at 79 chars. Therefore those that have UTF-8 chars (2 or more bytes) become shorter when wrapped. The more UTF-8 chars a line
has, the shorter.
This is noticeable in readers that don't support flowed text (most NNTP clients).
ASCII:
aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou aeiou
UTF-8:
áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú áéíóú
You're right about JamNNTPd not supporting multibyte. It wraps lines
at 79 bytes, not at 79 chars. Therefore those that have UTF-8 chars
(2 or more bytes) become shorter when wrapped. The more UTF-8 chars a
line has, the shorter.
problem with jamnntpd is that it does not support multibyte,
You're right about JamNNTPd not supporting multibyte. It wraps lines
at 79 bytes, not at 79 chars. Therefore those that have UTF-8 chars
(2 or more bytes) become shorter when wrapped. The more UTF-8 chars a
line has, the shorter.
On 25.02.2024 13:25, Carlos Navarro wrote:
Actually by default Jamnntpd wraps to 72, not 79, but that is irrelevant. :) As you said, the problem with utf-8 text is that Jamnntpd counts bytes, not chars.problem with jamnntpd is that it does not support multibyte,You're right about JamNNTPd not supporting multibyte. It wraps lines
at 79 bytes, not at 79 chars. Therefore those that have UTF-8 chars
(2 or more bytes) become shorter when wrapped. The more UTF-8 chars a
line has, the shorter.
I've set "#define WRAP_WIDTH 972" now in this Jamnntpd. I can re-wrap quoted text with Ctrl-R in TB. Seems nice so far.
'Tommi
...0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789 0123456789
'Tommi
Hello, Tommi Koivula.
On 01/03/2024 8.51 you wrote:
On 25.02.2024 13:25, Carlos Navarro wrote:
problem with jamnntpd is that it does not support multibyte,
You're right about JamNNTPd not supporting multibyte. It wraps lines
at 79 bytes, not at 79 chars. Therefore those that have UTF-8 chars
(2 or more bytes) become shorter when wrapped. The more UTF-8 chars a
line has, the shorter.
Actually by default Jamnntpd wraps to 72, not 79, but that isirrelevant. :) As you said, the problem with utf-8 text is that Jamnntpd counts bytes, not chars.
I've set "#define WRAP_WIDTH 972" now in this Jamnntpd. I canre-wrap quoted text with Ctrl-R in TB. Seems nice so far.
'Tommi0123456789 0123456789 0123456789 0123456789 0123456789
...0123456789 0123456789 0123456789 0123456789 0123456789
'Tommi
http://pics.rsh.ru/img/Screens5769_saapazgl.jpg
Looks ok now in Hotdoged.
Looks ok now in Hotdoged.
http://pics.rsh.ru/img/Screens5769_saapazgl.jpg
Looks ok now in Hotdoged.
Definitely doesn't look ok quoted back by hotdoged though. In TB or Golded. :)
From what I've gathered, WRAP_WIDTH is your quotes. LINE_WIDTH is what
you write.
You probably don't want to mess with stuff you've quoted, as that has to
be viewable in a 80 column terminal (BBSes, for example).
LINE_WIDTH may need to be adjusted to 72, instead of 79. So when quoting
79 characters it won't wrap the last 7 characters on to the next line.
Actually by default Jamnntpd wraps to 72, not 79, but that is
irrelevant. :)
TK> Actually by default Jamnntpd wraps to 72, not 79, but that is
TK> irrelevant. :)
True. O:-)
I've found that JamNNTPd wraps at 72 chars in all lines except the last one, that can be up to 79 chars.
Now I'm testing WRAP_WIDTH = 72 as well as LINE_WIDTH = 72. But I have
a feeling my long tearline will wrap early now, which I can't see
until I post this message.
Regards,
Nick
... "Take my advice, I don't use it anyway."
--- Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:115.0) Gecko/20100101 Thunderb
* Origin: _thePharcyde distribution system (Wisconsin) (1:154/10)
Now I'm testing WRAP_WIDTH = 72 as well as LINE_WIDTH = 72. But I have a feeling my long tearline will wrap early now, which I can't see until I post this message.
I've found that JamNNTPd wraps at 72 chars in all lines except the
last one, that can be up to 79 chars.
Yes, and it seems that it always wraps the text it shows to the
client. Defined by WRAP_WIDTH in nntpserv.c.
I've found that JamNNTPd wraps at 72 chars in all lines except
the last one, that can be up to 79 chars.
Yes, and it seems that it always wraps the text it shows to the
client. Defined by WRAP_WIDTH in nntpserv.c.
JamNNTPd always sends wrapped text. If flowed=on, it will append a
space to the end of each wrapped line. It's up to the client (if it
supports format=flowed) to re-join those lines and show it as a
paragraph.
However, in this server the Content-Type line has no "format=" part at all:
Content-Type: text/plain; charset="iso-8859-1"
So, IMHO the "hard wrap" setting WRAP_WIDTH 72 is not working as it
should with "flowed on". I think Jamnntpd should adjust the WRAP_WIDTH setting according to the "flowed" setting and not wrap lines with
"flowed on".
So, IMHO the "hard wrap" setting WRAP_WIDTH 72 is not working as
it should with "flowed on". I think Jamnntpd should adjust the
WRAP_WIDTH setting according to the "flowed" setting and not wrap
lines with "flowed on".
If I'm understanding you correctly, WRAP_WIDTH should remain 72 only
when 'def_flowed=off', and when 'def_flowed=on' it should be disabled completely? If that's the case, I agree.
In format=flowed, long lines are wrapped but they have a trailing space.
The client, if it supports this format, joins the lines, showing them as
a paragraph.
If I'm understanding you correctly, WRAP_WIDTH should remain 72 only
when 'def_flowed=off', and when 'def_flowed=on' it should be disabled
completely? If that's the case, I agree.
In format=flowed, long lines are wrapped but they have a trailing space. The client, if it supports this format, joins the lines, showing them as
a paragraph.
In format=flowed, long lines are wrapped but they have a trailing space.
The client, if it supports this format, joins the lines, showing them as
a paragraph.
Yep.
But for some reason TB doesn't act like this when using jam/smapinntpd.
In format=flowed, long lines are wrapped but they have a trailing
space. The client, if it supports this format, joins the lines,
showing them as a paragraph.
Yep.
But for some reason TB doesn't act like this when using
jam/smapinntpd.
In format=flowed, long lines are wrapped but they have a trailing
space. The client, if it supports this format, joins the lines,
showing them as a paragraph.
I understand this. However, what Tommi and I are discussing is that jamnntpd/smapinntpd is not doing this properly. I'm not sure
'WRAP_WIDTH 72' is very constant, as I see wrapping all over the place between 72-79 characters (which sometimes even wraps a word to the
next line and doesn't include a quote character before it). So either
the 'WRAP_WIDTH 72' and 'LINE_WIDTH 79' settings are messing with each other, or they're not implemented properly.
So, in my questions to Tommi, I'm more or less asking if anyone has actually figured out what WRAP_WIDTH is actually doing. It is only
checked for once in the code, and so is LINE_WIDTH. Neither of them
are checked for in the format=flowed section. It almost seems like the format=flowed section was an afterthought, and was added in after what
it was already doing. *shrug*
On 4/4/24 07:57, Carlos Navarro wrote:
03 Apr 2024 21:16, you wrote to me:
>> In format=flowed, long lines are wrapped but they have a trailing
>> space. The client, if it supports this format, joins the lines,
>> showing them as a paragraph.
TK> Yep.
TK> But for some reason TB doesn't act like this when using
TK> jam/smapinntpd.
Works fine for me:
https://www.cyberiada.org/fido/tmpfiles/flowed-wide-window.jpg
https://www.cyberiada.org/fido/tmpfiles/flowed-narrow-window.jpg
Very interesting...
When JamNNTPd sends to the client,
- lines that are 79 or less are not wrapped
- lines longer than 79 are wrapped by 72
Strange as it may seem, that's how format=flowed is defined in the RFC.
Nevertheless, those are the suggested values (that JamNNTPd uses). Other numbers could be used instead, as long as they are 78 or less.
The code block for format=flowed appends spaces to the end of wrapped lines and does the 'space stuffing' at the beginning (inserts extra
space if it begins with space), as defined in the RFC.
I stand corrected. After resetting all wrap and flow custom settings in TB, text seems to flow just fine. :)
TK> Yep.
TK> But for some reason TB doesn't act like this when using
TK> jam/smapinntpd.
Hello Tommi,
On Thu, Apr 04 2024 20:50:50 +0000, you wrote:
I stand corrected. After resetting all wrap and flow customsettings in
TB, text seems to flow just fine. :)
Nevertheless, those are the suggested values (that JamNNTPd
uses). Other numbers could be used instead, as long as they are
78 or less.
Ok. So it's doing what it should be doing, then?
The code block for format=flowed appends spaces to the end of
wrapped lines and does the 'space stuffing' at the beginning
(inserts extra space if it begins with space), as defined in the
RFC.
Why would you want to stuff spaces at the beginning of a line if it
starts with a space? What is the reasoning behind that?
I think the problem comes in to play with this example:
Quoting a 78 (78 is just an example, it could probably be anywhere
from 74 or 75 or more characters) character line with smartquote will
not reformat the line back to 78 characters (or within the 79 line
width limit it should be at). It will then append the space, initials, quote character, and another space.. now making it 4 or 5 characters longer. This will then wrap the last word to the next line and it
won't have a quote character in front of it.
I would imagine it would also do this without smartquote enabled,
however it wouldn't add as many characters to the line, so you would
see the issue less (only when a line hits 77 or 78 characters
probably).
Golded seems to handle this, but Golded also seems to support
displaying quoted lines longer than 79 characters, which may very well
be because of the issue(s) above.
On 4/4/24 14:53, Tommi Koivula wrote:
On 4/4/24 07:57, Carlos Navarro wrote:
03 Apr 2024 21:16, you wrote to me:
In format=flowed, long lines are wrapped but they have a trailing
space. The client, if it supports this format, joins the lines,
showing them as a paragraph.
Yep.
But for some reason TB doesn't act like this when using
jam/smapinntpd.
Works fine for me:
https://www.cyberiada.org/fido/tmpfiles/flowed-wide-window.jpg
https://www.cyberiada.org/fido/tmpfiles/flowed-narrow-window.jpg
Very interesting...
I stand corrected. After resetting all wrap and flow custom settings in TB,
text seems to flow just fine. :)
'Tommi
---
* Origin: smapinntpd/lnx (2:221/1.0)
| Sysop: | Coz |
|---|---|
| Location: | Anoka, MN |
| Users: | 2 |
| Nodes: | 4 (0 / 4) |
| Uptime: | 07:19:05 |
| Calls: | 99 |
| Files: | 4,658 |
| Messages: | 216,653 |
