Tales from the Countryside

Sorry for the long waits between posts. Not only have I been busy with work trips to the U.S. (gained some quarter-pounders of weight, but worked on some cool Pea combines), it just gets harder and harder to write about something I didn't write about before - almost 200 posts further! Graphics, physics, Programming / Delphi, games, horror, complaints about this and that... I like writing, but the topic needs to be fresh, funny, or informative. Not just for the sake of writing.

So, stirring my hand in a somewhat empty "idea grab-bag”, what do we got today? Well, why not tell a bit more about that other stuff I use to program most days of the year; agricultural machinery? No, it has nothing to do with games, but for you programmers out there, it might be interesting to have a peek into the industrial/vehicle area of the wide programming spectrum. Very different disciplines and techniques on one hand, but also the usual programming/design challenges at the other hand, like making a more abstract, reusable framework as you would do with game-engines. And also for you non-programmers, the story below should give some brief technological insights of our underestimated country-brothers and sisters

That feels right, doesn't it?

Alien Farms

As for me, other than eating steak and sometimes having the heavy odours of sugar beets or pigs rolling over our village, I don't have any agricultural background. I still know milk comes from a cow, drank beer in barns, but that's about it. As a kid, I associated tractors with mud, manure, and "that other world", somewhere outside the village.

It's a bit strange that  -the Netherlands being a very small and dense packed country- having farms just around the corner, there is an invisible iron curtain between "those from the city", and "those others from the countryside".  Farms with both cattle and crop are everywhere out there, yet somehow hidden between our cities and villages. You don't see heavy equipment or John Deere dealers at every corner as you would see in America. There are very little agricultural oriented schools, and back at school I knew very few farm-kids. Usually they would blend in unnoticeable, or form their own little group of little bulky blonde hair guys with old fashioned clothes and that funny accent, making them sound a little bit dumber than the rest of us. But even more dumber are kids nowadays that actually believe milk and apples are produced in factories; they never visited a farm.

That image changed when I had to find a final school assignment. Browsing through a list of IT companies, one name jumped out directly, saying "Ploeger",  which means "Somebody plowing". 99 out of 100 IT vacancies were about testing some unknown stuff, or web-development. Hell I like programming, except the web-part of it, and neither did I vision myself in a suit, drinking coffee in some shitty office with other boring dudes. So that particular name caught my interest directly... I was just guessing it had something to do with tractors, those things I associated with poop, dirt and a certain degree of dulness. But which in all of a sudden seemed to be like a better alternative for me. Real men, dirty hands, oil, chewing tobacco, handling heavy metal. Yeah...

So I googled for that name, of course it could just as well be some little 3-men company making a database for the local supermarket. But my instinct was right; not tractors but even bigger: self-propelled (meaning not towed but self-driving) harvesters, combines. Exactly what I was hoping for. Not much later I got my assignment; update the display + "computers" that run the whole circus in such a machine, a Pea combine in this case. Ploeger would also produce Potato, Bean, Spinach, Parsley and Flower bulb harvesters by the way.

Blackbox beat

I stranded in a quite different world. Not talking about farms and manure, but the type of programming-work. Basically there are two types of boys; those who tinker their first moped, and those who don't. I definitely belong in the second category, meaning I can't hold a hammer (no I’m not a plumber), nor have knowledge about anything motorized really. Well, let's say every person

has its own talents. Yet it's a bit ashaming you enter a factory where it's all about welding, hydraulic oil, diesel fumes, and Auto CAD drawings. You'll be programming a moving vehicle, but you don't even know the basic principles of combustion in an engine.

And almost 10 years later, I still don't know honestly. Not that people never explained me, or that I'm not interested. It's just that my head doesn't store things unless I use it in practice 100 times. The good news though, you don't really have to. Every piece of knowledge is valuable of course, but in general industrial applications are perfect examples of a "Black Box". There is input (sensors), some abracadabra happening inside that computer, and then there is output (actuators). One our machines that would typically mean. I’ll explain.

Input: Electrical sensors

Dashboard, joystick, and other cabin switches or potentiometers (knobs you can turn), to control a machine. Sensors that measure speed (rotating shafts that generate pulses), (fluid)temperatures and levels, mechanical positions determined via angle or proximity (contact yes/no) sensors. Sensors usually provide an electrical "Digital" or "Analog" signal. Digital simply means On or Off (0/1), like pressing a switch. Analog signals will gradually increase or decrease a voltage, current or resistance value. For example, a laser-distance sensor could deliver a signal between 500 and 4500 millivolt. Anything lower or higher could indicate a failure (no signal / short-circuit), and anything in between can be rescaled to centimetres. 

A possible sensor scale function. A signal of 2.5Volt would mean 75 cm in this case. Very low or high signals indicate (electrical / wiring / connectiong ) issues, and we shouldn't rely on this sensor then. Never ever assume stuff keeps working forever on a machine.

Sensors are used to inform users, and/or to define a state of a (sub)system, such as a position. You can put a sensor on anything really, although it would increase the price, as well the complexity of a machine. More electronics = a smarter machine, but also more chance on failure & more searching whenever things go wrong. And things WILL go wrong. I bet you heard your uncle complaining about how he could fix his old car by bonking it with a hammer, and nowadays some expensive nerd has to attach his laptop for every fart. Well nice to meet you, I'm that nerd.

Sensors are used to inform users, and/or to define a state of a (sub)system, such as a position. You can put a sensor on anything really, although it would increase the price, as well the complexity of a machine. More electronics = a smarter machine, but also more chance on failure & more searching whenever things go wrong. And things WILL go wrong. I bet you heard your uncle complaining about how he could fix his old car by bonking it with a hammer, and nowadays some expensive nerd has to attach his laptop for every fart. Well nice to meet you, I'm that nerd.

Input: CAN-bus

Another type of input are digital messages, carrying the outcome of another system. For example, a GPS sensor that measures coordinates. Often such (advanced) data is provided via a communication interface, as an ordinary electrical signal is insufficient. You probably have heard of the good old "COMport" (remove the word "good" here), and otherwise you know about USB. An USB mouse is basically and advanced input, communicating data like a position, movement and clicks through that wire.

On Vehicles, we usually have more than one computer device. It's not that we put a desktop PC somewhere in the cab, running the whole show. Smaller, less powerful, but very robust "Controllers" are placed throughout the machine -often physically nearby their target hardware, such as the Engine, Cooling-fan, or dashboard. Because of limited hardware capabilities, as well as making reusable standard components, Controllers usually target just a few specific functions, rather than controlling the entire machine. So in other words, you'll end up with a network of multiple, complimentary modules.

Now on our machines, we make most stuff ourselves (except Engine / Powertrain units). But on cars or trucks -made by millions- the industry relies on "from-the-shelve" components; Engine could be made by Mercedes, whipers by Bosch, dashboard console by Arnold Schwarzenegger. The real challenge: how to ensure all these different nodes can talk with each other?

On vehicles, most communication between those Controllers is done via CANbus. CAN stands for "Controller Area Network", and was developed a few billion years ago. 1983 to be more precise, by Bosch, although it would take another 10 years or so before the automotive industry really started adopting CAN as a standard. What TCP/IP, WiFi and Ethernet are for modern computers, that's CANbus for vehicles. A simple, 2-wire, robust network that allows multiple nodes(also called Controllers, ECU's, Modules, Devices, --> programmable chips) to chat. A bit like a modern network, but much slower, yet also easier. It doesn't require hubs or routers, nor do nodes require an address, although practical situations usually require each node to have at least an unique number.

CAN works on the principle of broadcasting small data packets. Typically each node measures or calculates a bunch of numbers and states. Conveyor belt speed = 200 RPM, Running = True, lowSpeedAlarm = False. Those numbers are tightly packed into an "Envelope", which can carry up to 8 bytes (64 bits), which usually offers room for 8,4 or 2 numbers, and/or a bunch of on/off states. Not much, and certainly not suitable for multimedia streams or text-approached messaging (like XML), but at least you can send multiple envelopes, and give them an unique messageIDs.

Nodes will broadcast data envelopes every X (milli)seconds, or on request/event. Other nodes, such as a Display, can read those packets, and do whatever they need to do with that data. As said, envelop messageIDs, and sometimes node addresses are used so a receiver knows what to expect in envelopX from nodeY. It's truly simple, and that's exactly what we need in vehicles, where robustness (and thus simplicity + reliability) is mandatory for its safety. Nodes going offline because of some IP address conflict is absolutely unacceptable.

The lay-out and formatting of messages is up to you... but to avoid a random mess, manufacturers create their own protocols, and/or follow Industry standards, such as J1939 or ISObus.

Output

Last but not least, there shall be Output. This will put things in motion. On the computer side, we have electric output pins. They can deliver low-power signals, again in Digital or Analog format. Digital signals could be 0 or 24Volt for example, which is just enough to drive a small light, or to switch a Relays. Analog outputs could be currents, voltages, or quick-pulsating signals (PWM) that gradually increase (or decrease) as you desire "more".

"More" translates into physical systems that can make things go faster, harder, louder.

Such systems are typically using pressurized hydraulic oil, pneumatics (compressed air), or high-power electricity. Servo motors can turn a weak signal into a powerful and precise displacement of shafts. Proportional pumps or valves can regulate air or fluid flows, to drive shafts or push/pull cylinders. On agricultural machines, most of the work is done via hydraulics. A little 12 or 24V signal is enough to power a relay, that opens a valve, letting through oil that can push many thousands of kilograms.

Finally, just like input doesn't necessarily have to be a physical, electrical signal, nor does output. Systems could send out (CAN) messages with computed results, designated for another system. For example, a Combine could broadcast its (GPS) position, state, groundspeed and settings, so the fleet manager can monitor his machines on a computer. This is what they call "Telematics" by the way. Another example is the Engine ECU. Besides regulating the actual engine components, it measures all kinds of things, like coolant temperature or oil pressure. Other systems like the Display use this to warn, to drive cooling fans, or destroke motors if the engine loads tends to get too high. The Engine doesn't know who/what/how its signals will be used, but it just drops them on the bus, just in case somebody needs them.

Once upon a time, an ox or even people did all the work. Now people's job is not to fall asleep behind an (almost) self-driving machine where hydraulics are the muscle, and computers the brain.

The Blackbox part

Between the Input and Output sections, there is the "Blackbox". The microchip & program

part. I don't know why they call it Blackbox, but it probably it has to do something with the fact that 99% of the people have absolutely no clue what happens inside that box. And even if you were a programmer, you still don't know unless you have seen the documentation or actual program code.

Your typical industral programmable Controller (see on the left) depicts the term "Blackbox" quite well. No idea what happens inside. Remember, they don't come with displays either! They emptied the Switch Cabinet (right) greatly though; lot's of relays, transistors, timers and PIDs are now in the software. 

Yep, that's where we shine girls. Whether you'll be programming phone apps, games, tractors, or a webshop, programming will always be about logic, algorithms, some math, and trying to make systems flexible enough so they can be easily adjusted or ported to another application. Yet, my experience is that industrial programming is still quite different though. For one thing, you won't be programming in a traditional style, as you would do with Java or C++. Or at least, you can, but it's just not common. And for a very good reason. Machinery often uses PLC's & Ladder diagrams. Vehicles and Hydraulic application are often using CoDeSyS, Guide Plus1 from Sauer Danfoss, or (my favourite) IQAN from Parker Hannifin. The latter two are major suppliers in hydraulic equipment, and also developed their own range of controllers, displays, and design software.

In all those cases, the design software provide a far more visual way of programming. It often looks more like an electrical drawing, instead of text based code lines. Being in a very specific segment (vehicles, hydraulics, CANbus), these tools are also far more limited and restricted. That means you can do much more in C++, but there is also much more that can go WRONG in C++. All the "building bricks" that Siemens, Danfoss, Parker, Hitachi, Omron or whatever industrial supplier provide in their software, are robust and proven. You really don't want your Boom Crane tumble over because of some rare pointer error caused by a once-in-a-million multi-threading mistake. Traditional programming languages are far more powerful, but also require far more energy and skill to develop & test your own "building bricks". The average programmer is just not capable of doing that, and even so, it would cost companies a lot of time and money.

Here three industrial programming tools. Quite different huh? They have one thing in common though; take away advanced traditional programming techniques that are bound to go wrong, and offer ready-to-go blocks that do the hard stuff for you.

It took me some time and pride to step away from advanced (Delphi) to a more simple platform,

but once you start dealing with angry clients, broken machines, confused service personnel, or 8 year old machines that suddenly want a new feature, you're more than happy with these simplified systems. Really, my student "Look at me programming cryptic hard-core shit" attitude shifted

towards "Easy doez it" over the years.

The Blackbox part: PID regulators
Focus is not so much on awesome programming techniques, but on smart design and making stabile (Idiot-proof) "Regulators", which demands knowledge of the machine and its users. A Regulator typically uses some input (sensors), parameter (user settings), to calculate an output signal that goes to a relays, pump or valve (see "Output" chapter).

Example. You may have heard of "Closed-Loop systems" or "PID Controllers". A very common task in industrial applications, is to maintain a certain level of something. That could be a thermostat, trying to keep 21 degrees C in your living room. It could also be a proportional pump, that tries to drive a shaft at 150 RPM. A very simple system would require a setpoint (user wants 150 RPM), that rescales to a certain output. 1Volt = 0 RPM, 10Volt = 200 RPM, 7.5V = 150 RPM. That's theoretical though. If Hulk Hogan strangles the shaft, it may not turn at all, 7.5Volt or not. In practice, shafts may get dirty/rusty, valves or pump may differ a little each time, cold- or hot oil fluids will result differently, or an increase of product-load on the shaft will slow it down.

The best way to compensate this, is by using a "Closed-Loop system", which can be a PID regulator. A sensor is used to measure feedback -the actual shaft speed / temperature / level / whatever-. If the measurement is below our desired setpoint, the output should increase or start. And vice-versa, if the measurement is above our desired setpoint, the output should decrease or stop. You would stop an heater as soon as the temperature rises above the desired temperature. In fact, you should have stopped a whole lot earlier, because temperatures usually change very slowly, but once they, they keep dropping or rising for a while. In other words, their feedback is very indirect. PID Regulators often have their P(orportional) I(ntegral) and D(ifferential) factors adjustable, so you can make it more or less aggressive to differences or rate-of-change. The magic trick is to configure it in such a way, that it responds quick enough. Not too early, not too late, not too much, not too little.

If very accurate speeds aren't that important, or if a slowdown is highly unlikely, we stick with just theoretical output. An extra speed sensor & PID Controller = expensive. But if it does matter, there's work for us programmers. Not just making the PID Controller itself, we also need a place where we can change the setpoint (potmeter, display, ...), and some feedback is desirable. If the speed really gets too low (shaft stuck / actuator malfunction) we'd like to see an alarm. And depending on differences and accuracy requirements, you probably need some calibration procedures as well.

So résumé, we have a network of Controllers, and usually a Display. Each Controller receives input(sensors), either via directly connected sensors, indirectly connected sensors (on I/O expansion devices), or via CANbus. Controller regulates certain functions, such as shafts, conveyors, lights, steer- or driving systems. Finally we have output; Controllers form electrical signals, going to Relays, Servo's, Pumps or Valves. But Output could also be in the form of outgoing messages, picked up by other systems.

As said, as a programmer, the good news is that I don't have to care much about the Input and Output sections. And vice-versa, nor do the handy guys care about what happens inside that Black Box. Yet the interesting part on such machines, is that many different disciplines meet. The input section is mainly designed by electrical engineering, and executed by electricians. The output part by hydraulic engineering and (CAD) drawers, as dimensions and physical rules will apply here. And somewhere in between is the programming. On a helicopter-project-management level, one would have to understand bits from all departments. Especially where they thigh together.

So to finish my little story about hillbillies stuck in mud & time, I quickly learned that, beneath the layers of dirt, there is a wonderful world of technology. Hence, the modern farmer isn't some smelly old grumpy dude with rubber boots and a sickle. The modern farmer sits behind a computer, collecting data from charts, robots, weather forecasts, satellites, and so on. The modern farmer is a manager on all fronts. Crop quality, machinery, personnel, import/export, animals, service, ... A manager, but occasionally with some dirt on his hands, still knowing what hard work means.

Combines as we make, costs hundreds thousands of dollars, have a wide network of service points spread all over the world, contain a big deal of computers and touchscreens, carry about 100 to 200 I/O (inputs/outputs) on average, communicate with clouds to upload position and data, as well as to download settings or chart data for partially automatic driving. Tractors, trucks and combines are driving forces behind communication technologies like SAE J1939 or ISObus; committees trying to standardize the messaging between Controllers, engines, tractors and additional equipment. My little story above just scratched the surface. So next time when you see a tractor or hear a funny talking guy from the countryside, just remember there is much more than pigs and poop out there. And talking about poop… just started on a fertilizer machine. Looking forward to see, feel and smell it :)