CoolTerm xon/xoff problem

[joanlluch]

There is a possible workaround in CoolTerm for when XOFF is not being honored by the driver. How many received bytes can your Arduino code handle at once?

Thanks for your suggestion.

The 8 bit Arduino microcontrollers based on the AVR chips only have 64 bytes of incoming serial buffer. It's certainly not much, but it's more than enough for human typed text on a terminal. Even if the keystrokes processing on the Arduino side would be really slow, it's highly unlikely that a human would type text at such a fast pace as to completely fill the 64 byte buffer.

So the problem I have is not really on this scenario, but when attempting to send files over serial. In such case, you are right that slowing down the pace of transmitted data solves the issue. My files consist on lines of text terminated with CR characters, these lines contain commands that the Arduino parses and executes.

Specifying a delay on ColdTerminal based on the 0A (hex) character prevents loss of data due to Arduino serial buffer to overflow. This is a great feature on ColdTerminal by the way.

However, this of course slows down transmission, and its reliability may be inconsistent in cases where the Arduino takes longer to process the incoming data in general, or to execute a particularly difficult command in the text file. It's kind of a tradeoff between speed and reliability.

Ideally, a true flow control procedure would be a lot more desirable because it would be as fast as the Arduino allows, and fully reliable.

On macOS, I found a peculiarity that you may be able to use at your advantage.

So I tried to implement Xon/Xoff flow control on the Arduino side, and this certainly works on the CoolTerm side by preventing keystrokes.

But it doesn't seem to do anything to stop data transmission from a file. I'm unsure if this is a CoolTerm bug or just that the local outgoing serial buffer (in the computer) is filled too fast for the application to actually detect any Xoff on time (?).

I think it's not the latter because even with transmission delays in place, the application does not stop file transmission. The Xoff characters are clearly received and reported on the ColdTerm "Hex View" screen but they don't seem to do anything to stop file transmission?

Any ideas or suggestions ?

Thanks


*** EDIT ***

Ok, I found that ColdTerm DOES actually stop transmission when receiving an Xoff character. So it's NOT a bug.

However, for the whole thing to work, some transmission delay still needs to be specified in order to give ColdTerm some time to receive the XOFF character before it's too late. Otherwise, the computer output buffer seems to be filled too quickly and with too many characters before the XOFF action actually takes place.

I also optimised my code on the Arduino so that incoming commands from files are quickly stored on a temporary buffer, before the command is actually processed (as opposed to parsing incoming commands on the fly as more characters were received). This provides a consistent time from receiving a bunch of characters to the issuance of the Xoff character, which is totally independent of the command processing time, thus allowing for a much smaller 'delay' specified on ColdTerm and faster overall transmission.

So this seems to work fine for me now.

Thanks again

[roger]

Ideally, a true flow control procedure would be a lot more desirable because it would be as fast as the Arduino allows, and fully reliable.

Correct. And this should happen as close to the hardware as possible so that flow is stopped as soon as XOFF is received. That was certainly feasible when computers had actual, built-in, serial ports with on-board UARTs. I imagine this may be a bit more difficult to achieve with USB/Serial hardware and the device driver needs to do a lot more heavy lifting. Regardless, though, the end application (such as e.g. the terminal software) shouldn't have be to responsible for this.

Ok, I found that ColdTerm DOES actually stop transmission when receiving an Xoff character. So it's NOT a bug.

Correct, that's what the "Software Supported Flow Control" feature does. It's the last resort for when the hardware or the driver isn't controlling the flow and it just so happens that the end application can see the XOFF and XON characters. However, since software is slower than hardware, it takes a while for CoolTerm to detect an XOFF and pull the break.

However, for the whole thing to work, some transmission delay still needs to be specified in order to give ColdTerm some time to receive the XOFF character before it's too late. Otherwise, the computer output buffer seems to be filled too quickly and with too many characters before the XOFF action actually takes place.

By default, CoolTerm sends files in 256-byte chunks. I.e. in a single write operation, it sends more than the Arduino input buffer can handle. However, you can adjust this in the transmit options by reducing the packet size to a smaller number. Eg. 64 bytes, or perhaps 32 bytes. Also, instead of the line delay, you could use the packet delay option. This might make it possible to optimize it a bit.

I also optimised my code on the Arduino so that incoming commands from files are quickly stored on a temporary buffer, before the command is actually processed (as opposed to parsing incoming commands on the fly as more characters were received). This provides a consistent time from receiving a bunch of characters to the issuance of the Xoff character, which is totally independent of the command processing time, thus allowing for a much smaller 'delay' specified on ColdTerm and faster overall transmission.

So this seems to work fine for me now.

Great! I'm glad you got it working. I haven't really needed to use handshaking of any kind with Arduinos yet, so thanks for bringing this to the forum. Lot's of good information!

Have you done anything with RTS/CTS hardware handshaking? Deasserting CTS should also stop the flow if hardware handshaking is enabled. I wonder if the Arduino driver honors that.

Also, I'm working on the latest CoolTerm beta, and I just got done implementing support for XMODEM file transfer. The basic version uses a 128-bit packet size, so this may not be something that could be helpful with Arduino's unless the input buffer size can be increased from 64-bits to something a little larger (to accommodate for extra bytes such as checksum, etc) than 128 bits. Do you know if this input buffer size can be increased somehow?

[joanlluch]

Thank you for your reply. Everything makes sense, and I actually already did understand it correctly.

By default, CoolTerm sends files in 256-byte chunks. I.e. in a single write operation, it sends more than the Arduino input buffer can handle. However, you can adjust this in the transmit options by reducing the packet size to a smaller number. Eg. 64 bytes, or perhaps 32 bytes. Also, instead of the line delay, you could use the packet delay option. This might make it possible to optimize it a bit.

I didn't realise the packet size could be adjusted to and specify packet based delays. That's certainly a more generic way that will work with any kind of file, including arbitrary binary files with no explicit separators. I may try that as well, but since my files are carriage return delimited lines, the termination based delay works fine for me.

Great! I'm glad you got it working. I haven't really needed to use handshaking of any kind with Arduinos yet, so thanks for bringing this to the forum. Lot's of good information!
Have you done anything with RTS/CTS hardware handshaking? Deasserting CTS should also stop the flow if hardware handshaking is enabled. I wonder if the Arduino driver honors that.

No problem. As far as I know there's not such a thing as hardware handshaking in ATMega based Arduinos. However there's a "SoftwareSerial" library that implements a serial protocol in software, using standard Arduino pins, but I don't use it. Since that one is implemented in software, I don't see any reason why it could not support RST/CTS signals through additional pins if the developer chose to do so. But I really don't know. One caveat of that library is that it's slower than the native hardware, and consumes processor resources.

I tend to use controllers with multiple hardware serial ports when I need to. For example the ATMega32U4 in the Arduino Pro Micro board has two hardware serial ports, one of them runs through USB. My understanding is that the subject of hardware handshaking gets more complicated because of the USB protocol on top of the serial comms. I even need a MicroB to USB cable and then a USB to USB-C adapter, so that I can finally plug the Arduino board to my Mac. So I guess real setups tend to diverge from the old RS232 connectors and cables, and getting RST/CTS is near impossible if I understand it right.

Also, I'm working on the latest CoolTerm beta, and I just got done implementing support for XMODEM file transfer. The basic version uses a 128-bit packet size, so this may not be something that could be helpful with Arduino's unless the input buffer size can be increased from 64-bits to something a little larger (to accommodate for extra bytes such as checksum, etc) than 128 bits. Do you know if this input buffer size can be increased somehow?

Yes, the serial input buffer size of Arduinos can be increased, however this must be done by tweaking code in the Arduino core libraries. It's not possible to just call a function or do something in your own code to change that. This means that the solution is not portable, as it will only work on your local setup.

This old thread explains how that can be achieved without going to unnecessary talking: https://community.platformio.org/t/how- ... r-size/122