|
|
|
Arduino serial watchdog
Thursday, June 27 2013
I woke up early this morning and began a series of experiments to test out a solution to a problem I have been having. The solar controller in the basement, the homebrewed device that decides when it's time to run the circulation pumps, is, as you'll remember, based on Atmega328 microcontrollers running in the Arduino environment. It's connected to Woodchuck (my main computer) via a 100 foot long serial cable, which allows me to monitor its behaviour and rewrite its firmware whenever I feel like it. But there's a problem with this system, and it has to do with the way Arduino (at least of the generation I am using) reprograms its microcontrollers. When the IDE wants to reprogram the Atmega328, it asserts the DTR line of the serial cable, which forces the Atmega328 to reboot so that its bootloader program can slurp in and install the new firmware. The reason it does this is because the DTR line is connected to the Atmega328's reset line via a small (0.1μF) capacitor. While this works great in conventional Arduino environments, it's problematic when using a 100 foot serial cable. I've found that if the serial cable should become unplugged in the laboratory, that 100 feet of cable behaves like an antenna, pulling noise into the solar controller and causing the serial lines (most problematically, the DTR line) to wildly fluctuate through various states. This causes the solar controller to reboot itself endlessly, never staying up long enough to turn on any pumps. With some USB-to-serial adapters, the serial cable doesn't even have to be detached for these things to happen; they can even happen when Woodchuck is connected but hibernating. You can see how this might be a problem if I put Woodchuck to sleep and go on vacation for two weeks. What if the solar controller reboots itself endlessly that whole time? Not only will solar energy go wasted, but the hydronic fluid will boil out of the system and, though I have systems in place to recapture it, repriming the pipes is always an unpleasant job.
A possible solution to this problem would be to install a switch allowing me to disconnect the capacitor connecting the DTR line to the Atmega328's reset line. But then I'd have to go down to the basement and flip the switch on whenever I wanted to reflash the firmware. And then I'd have to remember to flip the switch off before going on vacation (or even just going to bed).
A better solution would be to have some sort of triggerable one-shot that could temporarily flip on an electronic switch connecting the DTR line to the capacitor going to the Atmega328 reset line. But how would I trigger that one shot? One possible way would be to transmit a certain sequence of characters down the serial line and have some microcontroller dedicated to looking for it. When it sees that sequence, it fires a one-shot that closes a switch connecting the DTR to the reset line via the capacitor, which leaves them connected for a period of time before disconnecting them. As an added bonus, nearly all microcontrollers are flexible enough to serve as both a one-shot and a serial line monitoring agent.
Before I could install any of this stuff, though, I had to do the experiments to demonstrate whether or not each element of system would be successful.
The obvious choice for a microcontroller capable of monitoring the serial line and then flipping an electronic switch was an ATtiny85; I'd ordered a bunch of them from Tayda electronics, where they'd cost $1.15 each. My Arduino environment is set up to allow me to program them directly from the IDE, and I have a little USB-based programming dongle. I also have a flexible homebrew environment for testing 28 pin Atmega microcontrollers, which has allowed me to work out the bugs in other solar controller systems. I could put the AtTiny-programming dongle next to the 28 pin programming system, wire them together, and see if there were any problems to be avoided.
It took no time at all to write the code to make an AtTiny behave as a programmable one-shot triggerable by a sequence of characters on a serial line. I ended up writing the code so that it would respond to serial character sequences of the form "!X[digit]" -- which would make the one-shot stay high for 10 raised to the power of [digit] seconds. Thus a sequence of "!X4" would cause the one-shot to stay high for ten thousand seconds. While I was at it, I also made the AtTiny be on the lookout for the sequence "!R" — if it saw that, it would pulse one of its few other lines for a hundredth of a second. This would allow me to hook that line to the Atmega328s' reset lines and perform a hardware reset, which is lowest level of reset there is (and impossible for the processor to mask). The AtTiny could thus serve as an external watchdog to independently control the environment of the Atmega328s. That's a lot of functionality for an eight-pin IC; given the cheapness and flexibility, it's hard to imagine ever using a 555 timer again.
Now I had to run some experiments to see if an RS-232 cable really is shareable by two microcontrollers. Initially it seemed that the answer was no, since I could not read any data from an Atmega328 when it was sharing both transmit and receive with an AtTiny. But if I removed just the AtTiny's transmit line from the Atmega328's transmit line, the Atmega328 behaved normally and could even have new software bootloaded into it. The AtTiny just sat there, listening (along with the Atmega328) to the RS-232 receive line, waiting for one of the two sequences it was supposed to act on. And when it heard one, it would either fire one of its lines as a variably-timed one-shot, or send a pulse down the other. Interestingly, whenever it received a message that was meaningful to it, the data coming from the Atmega328 from then on would be nothing but blank lines in the serial monitor. But that hardly mattered, because the serial display became good again the moment the Atmega328 rebooted, and there was no reason to tell the AtTiny to do anything unless an Atmega328 boot was imminent.
Now I had to run some tests to see how easy it would be to connect a capacitor between DTR and reset using a digital switch. The digital switch I wanted to use was one of the two bilateral switches not being used in a 4016 quad bilateral switch already present on-board. Research told me that the resistance through one of these bilateral switches when in the "on" state was about 300 ohms. Was that too much resistance? To test, I connected the capacitor from the DTR line to the Atmega328's reset line through a 300 ohm resistor and checked to see whether it was possible to bootload new data into the microcontroller from the Arduino IDE. It worked, suggesting that my entire system would be feasible.
So I brought the Solar Controller up from the basement, removed its hand-wired motherboard, installed a socket for the AtTiny85, and then wired it to the 4016, the Atmega328s' reset lines (both master and slave), and to the capacitor connected to the serial cable's DTR line. The whole system seemed to work correctly the first time I tested it.
serial watchdog schematic:
serial watchdog source code:
The way the front side of the Solar Controller is looking these days. Click to enlarge.
The way the back side of the Solar Controller is looking these days. Click to enlarge.
Later in the day, though, I was noticing some strange behavior from the solar controller. For whatever reason, sufficiency was being determined, but the relays in the older, 19th-Century part of the system were not firing, at least not for the hot water heating loop. I found myself opening up the relay box for the first time in years and staring at it with wonder. Who, after all, had any idea how all that stuff worked? There were two rotary switches and two relays hooked up to various cables in various ways, and it was even difficult to determine which wires carried low-voltage AC and which carried low-voltage DC (the high-voltage parts were mercifully hidden behind a protective plate of aluminium). I'd documented this stuff extensively, but hadn't bothered to label any of the wires, so it was all sort of a mystery. But if I was going to get mad, I only had myself to blame.
Using my multimeter, I traced a problem with all the way back to the zone valve controlling hot water from the boiler. But then I realized that that whole system was supposed to be overriden by a relay that is turned on by the Arduino-based solar controller in the summer time, and without the solar controller attached, I was going to have the problem I was seeing.
Eventually I decided that the problem must be the ULN2003 Darlington array, which is the device that translates Arduino-level signals into signals large enough to switch 24 volt relays. I tried swapping in different ULN2003s, and they'd seem to work for a bit, but then they too would fail or behave strangely. In some cases they would turn on all the relays but then fail to turn them off when the time came. To test all of these scenarios, I found myself adding new instructions to the serial data parser inside the solar controller. These allowed me to force sufficiency on the controller even when the temperatures from the sensors were too low.
By the end of the day, the nature of the problem was still a bit of a mystery, but I was strongly suspicious that I'd received a bad lot of ULN2003s, probably from Mouser.
By the way, I used to get most of my mail-order electronics from Mouser, but recently a new upstart called Tayda (referenced earlier) has appeared on the scene. Their parts are significantly cheaper (sometimes just a tenth the price of their Mouser equivalents). I have to assume they're operating at a loss until they develop a large customer base upon which to turn the screw. But until that day, they're my favorite supplier. Tonight I ordered ten ULN2003s, ten 40 pin strip headers, and twenty 8-pin DIP IC sockets and the price was only $6.35 with shipping and handling. The ULN2003s alone would have cost me $5.20 from Mouser, where shipping would have been $5 minimum in addition.
For linking purposes this article's URL is:
http://asecular.com/blog.php?130627 feedback
previous | next |
