• CPU Hog

    From deon@1:103/705 to Digital Man on Fri Aug 5 16:52:02 2022
    Howdy,

    Today I discovered SBBS with 2 threads both running at 98% CPU - having stopped and restarted SBBS, its all back to normal.

    At the same time, my fido hub alerted me to my system being "slow" and refusing packets. Curious, I looked through the logs and indeed it seems that when I polled my fsxhub, mail flowed normally, but when I polled my fido hub, it was stuck there receiving a file:

    Authentication successful:
    Attempting poll for node 3:633/280@fidonet
    JSBinkP/4 callout to 3:633/280@fidonet started
    Connecting to 3:633/280@fidonet at ftn633.vk3heg.net:24554
    Peer version: binkd/1.1a-115/Linux
    Will encrypt session.Authentication successful: secure
    Receiving file: /opt/sbbs/temp/e8c38a06.pkt (1.4KB)
    [no more output]

    It doesnt appeaer that this thread dies - (and it should time out after 5 minutes right?) I ran "binkit -p" and it's still stuck on this node after 20 mins.

    My fido hub is set to "Poll: Yes", so I'm suspecting everytime my system was polling, the old thread hadnt died yet, so eventually I end up with many threads tied up polling my fido hub.

    So, shouldnt it ultimately time out after 5 mins?

    Is there a way to "busy out" the polled node, so another thread doesnt try and call it again?




    ...ëîåï

    ---
    þ Synchronet þ Alterant | an SBBS in Docker on Pi!
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Gamgee@1:103/705 to MRO on Fri Aug 5 21:14:00 2022
    MRO wrote to deon <=-

    rig up a daily reboot. I do it for all my bbses. that will fix a
    lot of issues that may pop up.

    Hahahahahahahahaha! Not everybody uses Windoze, dummy.



    ... He does the work of 3 Men...Moe, Larry & Curly
    --- MultiMail/Linux v0.52
    þ Synchronet þ Palantir BBS * palantirbbs.ddns.net * Pensacola, FL
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From deon@1:103/705 to Digital Man on Sat Aug 6 12:34:16 2022
    Re: CPU Hog
    By: Digital Man to deon on Fri Aug 05 2022 01:07 pm

    Is there a way to "busy out" the polled node, so another thread doesnt try and call it again?

    The SBBS event thread is a single thread. BinkIt polling is normally done as a timed event, which is run as in the foreground of
    that single event thread. So I'm not clear what "another thread" would be.

    OK, so I have BINKPOLL (binkit -p) set to run 2 times per day.

    So are you suggesting that when SBBS runs it, it can never run a subsequent time, if the previous run never completes?

    My logs show it does run twice a day (not 12 hrs apart, ergo 2 times a day - but...)

    Aug 5 00:35:54 d-11-1 synchronet: evnt BINKPOLL Running native timed event: BINKPOLL
    Aug 5 14:36:58 d-11-1 synchronet: evnt BINKPOLL Running native timed event: BINKPOLL

    So if the first question is yes, then that would imply that binkit -p does eventually exit.

    I'm wondering then, what else could cause the CPU goes to 100% on 2 threads for an extend period of time? (I havent monoitored the time, but my vmware logs show it was pegged for 2 hrs before I intervened.)

    Since my fido link identified the cause (it seemed it was in his end), CPU is backed to < 5%.


    ...ëîåï

    ---
    þ Synchronet þ Alterant | an SBBS in Docker on Pi!
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From deon@1:103/705 to MRO on Sat Aug 6 12:35:30 2022
    Re: CPU Hog
    By: MRO to deon on Fri Aug 05 2022 06:53 pm

    rig up a daily reboot. I do it for all my bbses. that will fix a lot of issues that may pop up.

    Nah, I doubt it.

    I'm not a "reboot fixes it" guy. That may have been a windows strategy in the day, but I never thought that was the solution to issue.


    ...ëîåï

    ---
    þ Synchronet þ Alterant | an SBBS in Docker on Pi!
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Belly@1:103/705 to MRO on Sat Aug 6 19:27:08 2022
    Re: CPU Hog
    By: MRO to Nightfox on Sat Aug 06 2022 04:33 pm

    but don't listen to me, i have great uptime for decades and my shit always w

    I can't believe I'm actually replying to your drivel, but how do you have great uptime if you reboot nightly? Get your story straight.

    o
    (O)
    BeLLy

    ---
    þ Synchronet þ bbs.brazi.net þ www.brazi.net þ WARNING: May contain nuts
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From MRO@1:103/705 to Tracker1 on Sat Aug 6 21:16:20 2022
    Re: CPU Hog
    By: Tracker1 to DOVE-Net.Synchronet_Discussion on Sat Aug 06 2022 06:09 pm

    irc 0.07% 271.7MiB 6.91% 162MB / 739MB 20.5kB / 90.1kB
    bbsweb 0.11% 9.023MiB 0.23% 42.3MB / 110MB 500kB / 90.1kB
    ecweb 0.06% 282.9MiB 7.20% 205MB / 3.14GB 105MB / 467kB
    rmweb 0.06% 67.22MiB 1.71% 36.8MB / 246MB 28.1MB / 90.1kB doorparty 0.00% 5.055MiB 0.13% 29.2MB / 25MB 6.37MB / 0B

    System is mostly idle, most of the time. Nothing is really crazy, the
    web instances seem to use the most resources, which are kind of
    expected... Slightly surprised how much memory the IRC service uses on
    its' own, but not crazy either. I'd thought about breaking nntp off


    that's pretty sad because unrealircd uses almost no resources.

    dont forget to reboot.
    ---
    þ Synchronet þ ::: BBSES.info - free BBS services :::
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Tracker1@1:103/705 to DOVE-Net.Synchronet_Discussion on Sat Aug 6 18:09:42 2022
    As a minor followup... here's the docker stats for my running system, I
    think it's been up for about a few weeks since I last touched it.

    NAME CPU % MEM USAGE MEM % NET I/O BLOCK I/O
    sbbs 0.09% 185.1MiB 4.71% 540MB / 666MB 100MB / 2.44GB
    mail 0.03% 59.86MiB 1.52% 213MB / 347MB 2.82MB / 90.1kB
    ftp 0.00% 6.477MiB 0.16% 8.41MB / 10.6MB 8.19kB / 90.1kB
    svcs 0.08% 114.1MiB 2.90% 304MB / 48.3MB 6.38MB / 3.22MB
    irc 0.07% 271.7MiB 6.91% 162MB / 739MB 20.5kB / 90.1kB
    bbsweb 0.11% 9.023MiB 0.23% 42.3MB / 110MB 500kB / 90.1kB
    ecweb 0.06% 282.9MiB 7.20% 205MB / 3.14GB 105MB / 467kB
    rmweb 0.06% 67.22MiB 1.71% 36.8MB / 246MB 28.1MB / 90.1kB
    doorparty 0.00% 5.055MiB 0.13% 29.2MB / 25MB 6.37MB / 0B

    System is mostly idle, most of the time. Nothing is really crazy, the
    web instances seem to use the most resources, which are kind of
    expected... Slightly surprised how much memory the IRC service uses on
    its' own, but not crazy either. I'd thought about breaking nntp off
    from the rest of the misc services, since it's what I'm using myself
    most of the time, and was a little curious.

    VM Ram is 4GB total, and this is all that's running other than basic
    services and the OS ssh, etc.
    --
    Michael J. Ryan - tracker1@roughneckbbs.com
    ---
    ï¿­ Synchronet ï¿­ Roughneck BBS - roughneckbbs.com
    --- SBBSecho 3.15-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Tracker1@1:103/705 to deon on Sat Aug 6 18:20:06 2022
    On 8/4/22 23:52, deon wrote:

    Today I discovered SBBS with 2 threads both running at 98% CPU
    - having stopped and restarted SBBS, its all back to normal.

    What did your RAM usage look like? I've seen a similar issue a couple
    times, where I couldn't connect on the mail service port... I don't know
    what part of Synchronet was causing the issue... there was definitely
    some kind of memory leak as I was getting a lot of out of memory errors.

    I bumped the server to a higher memory option, then separated each
    service into a separate running instance... I haven't seen the issue
    since, though so no idea.
    --
    Michael J. Ryan - tracker1@roughneckbbs.com
    ---
    ï¿­ Synchronet ï¿­ Roughneck BBS - roughneckbbs.com
    --- SBBSecho 3.15-Linux
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From dragon@1:103/705 to MRO on Sat Aug 6 20:41:38 2022
    On 8/6/2022 16:56, MRO wrote:
    Re: Re: CPU Hog
    By: dragon to MRO on Sat Aug 06 2022 03:28 pm

    >
    > Some people are interested in WHY something breaks instead of just in a
    > quick "fix". It's called "debugging". I would think in your years of
    > experience you'd have heard of it. ;-)

    I absolutely know WHY something breaks normally. It's just good to reboot once a day to clear up issues that come up randomly. That's how bbses are. shit happens. We are usually running windows and launching legacy software made by hobbyist programmers. We can't be around all the time when something decides to go haywire and if a user can't connect one time they may never try again.

    You aren't a sysop, so you wouldn't know that; you just run a website that scriptkiddies scrape to attack us.

    ---
    � Synchronet � ::: BBSES.info - free BBS services :::

    Once again, your ignorance is showing. I ran BBSes (plural) for well
    over a decade on a variety of platforms. I never stopped running
    systems and services of all kinds since the late 80's and have made a
    good living at it.

    You keep making allegations about how my website is being used. Do you
    have any actual evidence, or are you talking out of your ass as usual?

    ---
    þ Synchronet þ IPTIA - bbs2.ipingthereforeiam.com:2323
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From dragon@1:103/705 to Nightfox on Sat Aug 6 20:43:56 2022
    On 8/6/2022 18:44, Nightfox wrote:
    Re: CPU Hog
    By: MRO to Nightfox on Sat Aug 06 2022 04:33 pm

    MR> but don't listen to me, i have great uptime for decades and my shit always
    MR> works.

    If you have it reboot every day, then it's not up continuously for more than a day. I wouldn't call that great uptime.

    I've been running a BBS since 1994 (with DOS at the time), and I didn't have to reboot my BBS machine every day to keep things working (and still don't).

    ---
    � Synchronet � Digital Distortion: digitaldistortionbbs.com

    Awww, I was going to say almost all of that!

    ---
    þ Synchronet þ IPTIA - bbs2.ipingthereforeiam.com:2323
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From fusion@1:103/705 to MRO on Sat Aug 6 23:33:00 2022
    On 06 Aug 2022, MRO said the following...

    you haven't been running it every day like i have.
    and i don't HAVE to reboot. I just do just to do some cleanup. shit happens with old doorgames and legacy programs.

    no, it doesn't.

    if you want to measure dicks, here's my vm host uptime. i only rebooted
    it because there was a network issue with ovh and i was trouble shooting.

    ovh.. you like flushing your hard earned money into the toilet?

    ... Multitasking: Reading in the bathroom

    --- Mystic BBS v1.12 A47 2021/12/25 (Windows/32)
    # Origin: cold fusion - cfbbs.net - grand rapids, mi
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From deon@1:103/705 to Tracker1 on Mon Aug 8 09:54:44 2022
    Re: Re: CPU Hog
    By: Tracker1 to deon on Sat Aug 06 2022 06:20 pm

    What did your RAM usage look like? I've seen a similar issue a couple
    times, where I couldn't connect on the mail service port... I don't know what part of Synchronet was causing the issue... there was definitely
    some kind of memory leak as I was getting a lot of out of memory errors.

    Yeah, I am confident RAM wasnt the issue.

    I've been running the same container config for 2+ years now, with the only different being an image update every now and again. (I can see from docker stats, that its happy with 168MB of 2G available to it.)

    The problem started when I switched to a new ISP, that uses PPPoE. As a result all of my IPv6 traffic was affected by the lower MTU, and while I addressed that for my server LAN, I'm assuming it is affecting the docker lan (havent definitively confirmed that yet, but it seems likely).

    Resetting the host (and the container) saw the CPU get back to 100% after a short while (with my fido hub trying to deliver me mail over IPv6). When he switched to IPv4, problem disappeared and CPU stays normal (< 5%).

    My initial report of this was due to the fact that when I polled him with binkit, it didnt time out after 5 minutes (which IIRC is the "normal" timeout - and I'm assuming since he is using binkd his side would have) and I was thinking that subsequent binkit -p polls were "adding" to the issue (together with him polling me as well).

    (And I dont think my side would have closed down when his side sent the TCP_FIN as a result of the binkd timeout, since there was (probably) other defragged packets that werent successfully delivered yet.)

    What's also strange, I've learnt that his side is also PPPoE, and I was only having this issue with his system - my other IPv6 connections were not affected.

    Not sure why a hung poll would also peg 2 cores as technically my side is waiting to receive - perhaps the repeated system call to check if there is data on the wire does result in this?


    ...ëîåï

    ---
    þ Synchronet þ Alterant | an SBBS in Docker on Pi!
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Nightfox@1:103/705 to Gamgee on Mon Aug 8 08:52:36 2022
    Re: Re: CPU Hog
    By: Gamgee to Jas Hud on Sun Aug 07 2022 09:16 pm

    but if you're running a bbs, and you have dos games and shit you
    should be running it on windows 32bit instead of emulating shit
    in a linux environment. it's easier and it's the right tool for
    the job.

    It's quite easy to run that stuff on Linux if you know what you're
    doing, which you clearly don't. It's simple. It works perfectly. It doesn't require a daily reboot to keep it running.... Hahahahahaha

    I switched my BBS over to Linux not too long ago, and to be fair, it seems there are some DOS doors (in particular, TradeWars 2002) that don't run in DOSEMU 1.x. I've heard it does run in DOSEMU 2, though I have yet to successfully get DOSEMU 2 working with Synchronet.

    However, regarding mro's statement about emulating stuff, I think even a 32-bit Windows uses some kind of emulated environment anyway. Windows has its NTVDM, the Virtual DOS Machine, though I don't know all the details of how it works. I do know that 64-bit x86 processors are able to run 16-bit binaries when running in 32-bit mode though..

    ---
    þ Synchronet þ Digital Distortion: digitaldistortionbbs.com
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Nightfox@1:103/705 to fusion on Mon Aug 8 15:38:20 2022
    Re: Re: CPU Hog
    By: fusion to Ryan Fantus on Mon Aug 08 2022 05:12 pm

    ntvdm64 is a software emulator they borrowed from windows for.. powerpc?

    As far as I can tell, NTVDM64 is meant to work on 64-bit Intel/AMD processors. Where did you see that it was for PowerPC?

    ---
    þ Synchronet þ Digital Distortion: digitaldistortionbbs.com
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)
  • From Ryan Fantus@1:218/820 to fusion on Tue Aug 9 00:30:20 2022
    another neat thing is each ntvdm gets it's own "640k" of ram, it's own EMS/XMS, etc.. it's really the scaffolding of the single-pc multinode
    bbs.

    Wow, I had no idea. That's pretty neat. Honestly, I use Windows 7 32 bit for my doorgame host and it works astonishingly well. No slowdowns or any other apparent issues. Only some games like Yankee Trader don't work; everything else seems to hum along as expected.

    it's neat stuff. bit of a shame the 16bit mode is not accessable from 64bit oses.. but we're lucky it works in 32bit mode. in theory it should be faster than the ntvdm64 one, but with modern cpus i don't know if you could measure that difference anymore

    Yeah, agree. I understand the reasons to delete legacy code but it does really put a thorn in my side, hehe.

    --- Mystic BBS v1.12 A48 2022/07/11 (Linux/64)
    * Origin: m O N T E R E Y b B S . c O M (1:218/820)
  • From Tony Langdon@3:633/410 to fusion on Tue Aug 9 19:52:00 2022
    On 08-08-22 19:49, fusion wrote to Nightfox <=-

    On 08 Aug 2022, Nightfox said the following...

    As far as I can tell, NTVDM64 is meant to work on 64-bit Intel/AMD processors. Where did you see that it was for PowerPC?

    an excerpt from the ntvdm wiki:

    Since virtual 8086 mode is not available on non-x86-based processors
    (more specifically, MIPS, DEC Alpha, and PowerPC) NTVDM was instead implemented as a full emulator in these versions of NT, using code licensed from Insignia's SoftPC.

    I was led to believe that it was possible to create a DOS VM, using the hardware virtualisation capabilities of modern processors, so theoretically there is a way around the lack of virtual 8086 mode in 64 bit mode at the hardware level, if I'm right.


    ... Does the name Pavlov ring a bell?
    === MultiMail/Win v0.52
    --- SBBSecho 3.10-Linux
    * Origin: Freeway BBS Bendigo,Australia freeway.apana.org.au (3:633/410)
  • From Tony Langdon@3:633/410 to Ryan Fantus on Tue Aug 9 19:58:00 2022
    On 08-09-22 00:30, Ryan Fantus wrote to fusion <=-

    another neat thing is each ntvdm gets it's own "640k" of ram, it's own EMS/XMS, etc.. it's really the scaffolding of the single-pc multinode
    bbs.

    Wow, I had no idea. That's pretty neat. Honestly, I use Windows 7 32
    bit for my doorgame host and it works astonishingly well. No slowdowns
    or any other apparent issues. Only some games like Yankee Trader don't work; everything else seems to hum along as expected.

    That's the magic of virtual 8086 mode. Same happened with DESQview on a 386, every session got its own 60k (less any OS/TSR overhead loaded before DV). And OS/2 really made good use of virtual 8086 mode with its DOS subsystem - the "better DOS than DOS".

    it's neat stuff. bit of a shame the 16bit mode is not accessable from 64bit oses.. but we're lucky it works in 32bit mode. in theory it should be faster than the ntvdm64 one, but with modern cpus i don't know if you could measure that difference anymore

    As I said in another message, I'm sure I read it's possible to spin up a DOS VM in 64 bit mode, by using the hardware virtualisation capabilities of modern AMD64 CPUs, which would result in similar functionality.


    ... Any safety factor set as a result of practical experience will be exceeded === MultiMail/Win v0.52
    --- SBBSecho 3.10-Linux
    * Origin: Freeway BBS Bendigo,Australia freeway.apana.org.au (3:633/410)
  • From Tony Langdon@3:633/410 to Daryl Stout on Thu Aug 11 19:59:00 2022
    On 08-09-22 16:22, Daryl Stout wrote to Tony Langdon <=-

    That was fun running a DOS based dial-up BBS under DOS 5 with
    DESQView, and experimenting with the QEMM Memory Utility. :P

    Yeah DV and QEMM were a big thing then, and tweaking QEMM was an art form. The automatic coinfiguration would get it close, but it was possible to get a few more kB with a tweak.

    However, Quarterdeck Software (which made DESQView and QEMM), as well
    as TeleGraphix (which pioneered RIP Graphics), are both long gone now.

    A bygone era, definitely.


    ... The Lab called... Your brain is ready!
    === MultiMail/Win v0.52
    --- SBBSecho 3.10-Linux
    * Origin: Freeway BBS Bendigo,Australia freeway.apana.org.au (3:633/410)
  • From Jared@1:103/705 to Daryl Stout on Thu Aug 11 12:19:12 2022
    Re: Re: CPU Hog
    By: Daryl Stout to Tony Langdon on Tue Aug 09 2022 16:22:00

    That's the magic of virtual 8086 mode. Same happened with DESQview on a 386, every session got its own 60k (less any OS/TSR overhead loaded before DV). And OS/2 really made good use of virtual 8086 mode with its DOS subsystem - the "better DOS than DOS".

    That was fun running a DOS based dial-up BBS under DOS 5 with DESQView, and experimenting with the QEMM Memory Utility. :P

    However, Quarterdeck Software (which made DESQView and QEMM), as well
    as TeleGraphix (which pioneered RIP Graphics), are both long gone now.


    DESQView and DOS 5 and while originally RA later Ezy was the pinacle of my 90s board time and ran Oglaroon (my original BBS name) for years, but I have to say nothing ever came close to the pre-emptive multitasking of my beloved Amiga 500.. not for many years. :D

    de VK2WAY

    ---
    þ Synchronet þ It's just a jump to the left (and a step the right)...
    * Origin: Vertrauen - [vert/cvs/bbs].synchro.net (1:103/705)