Achieving autonomy across industries and applications is vital to sustainability through lower carbon emissions, reduced waste, increased productivity and, above all, ensuring the safety of people. But how do we get there?
This episode of HxGN Radio features Sandy Kennedy, VP of innovation; Lee Baldwin, autonomy segment manager; and Neil Gerein, senior director of marketing from Hexagon’s Autonomy & Positioning division, explaining how assured positioning builds a foundation empowering autonomous solutions.
Sandy, Neil and Lee will each be speaking at HxGN LIVE Global as part of the Autonomy & Positioning Reality Summit in June 2022. To learn more about the Summit and to join us, visit https://hxgn.biz/APRSummit
SR: Hello, and thanks for listening to HxGN Radio. I’m your host today, Sarah Rowe.
Achieving autonomy across industries and applications is vital to sustainability through lower carbon emissions, reduced waste, increased productivity, and above all, ensuring the safety of people. But how do we enable an autonomous and sustainable future? Well, to get there, we need to ensure that autonomy is safe and positioning is assured.
In this episode of HxGN Radio, I asked three experts from Hexagon’s Autonomy and Positioning division about what assured positioning is, how it builds a foundation for safe autonomy and, ultimately, a sustainable future. These three experts are speaking at the Autonomy and Positioning Reality Summit at HxGN LIVE Global in June 2022. But, I’ll let them introduce themselves.
LB: I am Lee Baldwin. I’m a segment manager at Hexagon in the Autonomy and Positioning division. And at HxGN LIVE, I will be talking about automated road trains and how Hexagon is supporting to make road trains in mining autonomous.
NG: Hi, my name is Neil Gerein. I’m the senior director of marketing at the Hexagon Autonomy Positioning division, and I will be talking about anti-jamming and anti-spoofing protection of GNSS positioning.
SK: Hello, I’m Sandy Kennedy. I’m vice president of innovation with Hexagon Autonomy and Positioning. This year at HxGN LIVE, I’ll be presenting on sensor fusion and its applications to positioning.
LB: We use the generic term “GNSS” for global navigation satellite systems because there’s multiple ones. There’s the all familiar GPS. There is Galileo, which the European one. There’s BeiDou for the Chinese system. There’s GLONASS, the Russian system, and a bunch of local augmentation systems. And so all of those together create a set of satellite signal. So, when we talk GNSS positioning, it’s the positioning we get by measuring the distance to those satellites and using more than four of them to come up with a solution. When we say GNSS, we’re not specifically in that term including additional sensors. So, we’re not including inertial sensors, so it’s just pure GNSS positioning using those satellite signals.
SK: Assured positioning is not just a suggestion or a really good guess about what your position is. It is position information that you can really rely upon that you can be very confident is in fact true and representing where you are truly physically located.
NG: I really look at assured positioning as one of the pillars of automation and autonomy. In the context of autonomy, assured positioning is part of the overall autonomy stack. And when I say stack, that’s the software stack and all the sensors that go with it, composing those pillars of positioning, perception and planning for a vehicle’s autonomous system. So positioning is just as important as all those other ones when you’re talking about an autonomous machine.
SK: The inertial is one of the primary sensors that gets used to augment GNSS. So, an IMU, inertial measurement unit, those are accelerometers and gyroscopes and they’re measuring accelerations and rotations. That’s the function that is in your human inner ear, actually. If you’re sitting in a chair and you close your eyes and somebody rotates the chair, you can sense, you can feel, that you’ve been rotated even though your eyes haven’t seen the changing, you know, view in front of them. It is complementary in that its errors grow over time because it’s an integrated process, whereas GNSS sensors don’t really vary over time, so they’re a very synergistic combination.
It also doesn’t rely on anything external to the sensor. It is the inherent motion that’s being sensed. The only thing you really need to know is an approximation of Earth’s gravity field and Earth’s rotation to be able to use it. Otherwise, you can be in the dark, you can be underwater, you could be indoors, you can be in a completely obstructed GNSS environment, lack of light, all of those things. And because they are very contained like that, that also means that they’re difficult to tamper with or spoof.
Other information that can be used to augment GNSS are things like cameras or LiDAR or RADAR, something that’s going to be more perceiving the surroundings around you. And then often, if you have map information to go with that, you can then derive what your absolute position might be. You should definitely be able to tell how your relative position is changing with respect to fixed infrastructure around you.
So, yeah, inertial is the most common backup to GNSS. They both work at an absolute reference frame, so that’s helpful as well. You can use things like wheel sensor information if you’re in a vehicle, differential off of the wheels to infer how you’ve been turning and how fast you’ve been travelling. There’s a few other sort of, I guess, more limited sensors that can be used to augment the inertial and the GNSS.
NG: Assured positioning means: do we have a solution and do we know that it’s the correct solution? So, there is ways that we can lose GNSS positioning. It’s susceptible to interference so we could have an interfering signal, intentional or not intentional. Non-intentional might be you might have some interference in your local system, if it’s a clock line or some other electronics, and it might unintentionally jam GNSS. You might have an external interferer. Somebody is actively trying to jam you, deny your signal. Or they may be trying to spoof you, which is the idea they send a set of signals that look like the satellite signals, but they’re fake signals to make you seem like you’re at a different place or time.
Assured positioning means we have ways to recognise that somebody is trying to interfere or spoof your system. So first of all, am I getting the correct signal? And then on top of that we have mitigation, meaning that if somebody is trying to jam or spoof our signal, we can have a set of mitigation measures to make sure that we reject the jamming and spoofing so that we can have our true solution. So, our positioning is assured; we have the right position at the right time.
SK: When you’re a human and you know you’re looking at the blue dot on your phone, you have a lot of other redundant systems and, you know, sort of fallbacks if that blue dot is not believable or not available. It’s usually always available, but with like a really big uncertainty circle around it. So, if you’re a machine and you’re trying to operate 24/7, you must know where you are to, well, to achieve whatever task it is that you’re trying to do in physical space. Being lost is not an option. Not knowing where you are is not an option. And if you are a machine, you have less capability to infer where your position is or what’s around you than when you are human.
NG: Assured positioning certainly is very, very important for autonomous applications because in autonomous applications, the idea is to take the human out of the system. So, a good example is a rotary ring helicopter landing on the back of a ship. It’s a dangerous situation, especially if the sea state is there’s a storm going on. So, if we can remove the human from the loop, we can make it a safer system. But as soon as we remove the human from the loop, it means we need to rely on where we are and where we’re going, hence the navigation of GNSS. So that’s a case where you want to make sure that you know where the back of the ship is so you can land on the back of the deck. You want to know where the helicopter is, the rotary ring aircraft so it can go and approach the helicopter. And if something goes wrong, that somebody’s trying to jam it or spoof it, be able to alert a human in the loop, or be able to abort the landing so that you don’t cause an unsafe situation.
LB: When we talk about automation, it’s really very broad and encompasses assisted functions as well as autonomy or robots. And what I mean by assisted, this could be you use positioning in a vehicle today for your mapping or navigation system. And as those get more automated, those types of systems and ADAS systems get more automated in vehicles, they’ll be used prevalently and have to be more accurate. Assured positioning is important with these ADAS systems or automated driver assistance systems. And like ones in cars today, like lane keeping and automated, you know, cruise control, where it keeps you in your lane and the speed and everything else, assured positioning is even more important because you need to really know where that vehicle is within, you know, 10 centimetres. You don’t want drifting around all over the place. You want it to really keep it. And GNSS or GPS could be used in those types of scenarios. Most likely, you’d have a perception system to help, but you have to be centred on a lane.
SK: So additional sensors as this, like the example I used as a human, you know, it’s the way that if the lights are out, you can still manage to find your way to the bathroom in the middle of the night, for example, because you have more than one sensor that’s going to let you observe what’s around you. So, it’s either a fallback or it’s a way to double cheque or verify that the information that you’re getting from one sensor, you know, it’s consistent with what another sensor is telling you, for a redundancy cheque or if one sensor has failed because of the conditions and it’s not available anymore and you need another sensor to bridge that gap until you can have more than one sensor and then you can go back to redundancy and consistency and checking like that.
LB: The Society of Automotive Engineers came up with different levels, and you may have heard as SAE levels of autonomy. There’s six levels as they define them. And this was really for on-road. But it really applies everywhere. So, you have level zero, which is no automation. Driver does everything. And all the way up to level five, which is full automation. And that’s where the vehicle can drive itself in all conditions and the driver doesn’t do anything. It’s not even required. So, you have all these levels in between. So where does that leave us, where we’re at today? You go back to level one. Those are some of the more basic things that we’ve had today in the vehicle, basic cruise control, blind spot warning, and things like that. But the driver’s still in control.
Then you start to get a bit more with level two, automatic emergency braking and lane centring or adaptive cruise control. And I sort of look at Tesla’s autopilot as something in between level two and three because in level two, the vehicle can perform multiple driving tasks, but the driver’s always in control and has to be alert. So, if you look at level three, which is conditional automation, the vehicle can still perform the driving test, but the driver’s not required to remain alert. But if you’re in a Tesla doing autopilot, they still say you have to be alert. So, it’s somewhere in between.
So, if you look at it where we are today, it’s probably at level one to level two is where we’re at. And then level four, that’s higher automation. And that’s where companies like Waymo or TuSimple are really focusing on today. Now that’s on-road.
So from an off-road standpoint, in certain off-road applications like in mining, I’d say we’re probably at level four, but you don’t need a driver in there. Level five is where you take the driver out. They don’t do anything. Level four, it’s in certain conditions. So, I think that’s available today and those are driverless.
SK: Assured positioning is a requirement. It’s an absolute prerequisite for autonomy because a machine cannot be lost. It cannot be uncertain as to its location. That’s the primary difference between a machine and a human. Navigation capability is one of the fundamental characteristics of humans that sets us apart from other animals. So, you must know what your position is. You must know what your location is with full confidence to be able to make decisions and to take action as to what your next steps are. So, to have a completely, as are fully autonomous vehicle, a fully autonomous machine, it needs to be position aware. And if it is not position aware, that is one of the things that has closely examined in fault management and how that will be handled.
LB: It’s really being done in baby steps. You know, passenger vehicles have been at it for some time with initial cruise control that came out, I think, in the late ‘70s and all the way to where the cruise control now is quite automated, but it just takes time. And in an off-road environment, though, we’re already doing made the switch from automation to autonomy. And if you look at it, autonomous hauling, the very large ultra-class mining trucks, those are really level four autonomous today with no drivers. But those are controlled environments where they keep people out and they keep conditions. So, it’s basically happening today in certain environments, along with mining road trains. That’s kind of the next thing on long haul controlled access roads in certain areas of the world. You could do those things.
So really, it’s about kind of what you can control variables because on-road’s taking a lot longer based on those variables, because humans are very unpredictable as we know when you’re driving. So, the sensors and the software and testing has to be much more robust, let’s say. And then you have certain standards that come into play like functional safety. So, it’s just has to be at a much higher level to take care of on-road.
NG: Autonomy is really, really important in agriculture. You know, it’s one of the big markets we play in because the standard problem we’re always faced with in any sort of production system is we want to have as few as inputs to create as high as an output. And that means in the case of farming, that we don’t want to be wasting seed, we don’t want a waste fuel, we don’t want to waste fertiliser when we’re putting the crop in and also when we take the crop out.
So, the best way to assure that is to make sure that we’re going down the same lines on the field year after year after year, because that helps to make sure that we’re not over seeding or under seeding. We’re not using excessive fuel, and that leads to autonomy, to be able to take the human out of the loop. You have the human just do the turn at the end, to be able to monitor the system, but be able to rely on positioning and be assured of your positioning to make sure that that autonomous vehicle can go year after year, along the same crop path to maximise the output on the field. And so that just helps overall sustainability by reducing cost, reducing fuels, reducing effort.
SR: You’ve been listening to Sandy Kennedy, Neil Gerein, and Lee Baldwin discuss how assured positioning is vital in enabling safe autonomous applications and ultimately a sustainable future. These three experts will be speaking in depth about these topics during the Autonomy and Positioning Reality Summit, taking place in June 2022 as part of HxGN LIVE Global.
LB: At HXGN LIVE, I’m going to talk about automated road trains and specifically autonomous road trains and how Hexagon is making that a reality. What I’m going to do is go into considerations for developing and off-road autonomy system. I’m going to go into technical discussions about some of the subsystems like perception, planning, and positioning, how those can be solved and some of the considerations for off road use and some of the business drivers of why we’re doing road trains and why it’s important to miners and specifically more on some of the use cases where this technology could be used and applied to other industries besides mining.
NG: So, my presentation is going to go into some technical detail about hardware and software that we have that does jamming detection, jamming mitigation, spoofing detection, how you use spoofing mitigation, and the set of tools that we can use to be able to add onto that solution to make a very robust application so that our customers can, in turn, trust on us to provide the positioning that they need so that they can carry on and add value in their missions.
SK: So, for my presentation at HxGN LIVE, in-person fully in 2022, I’ll be talking about sensor fusion and how it’s applied to the positioning problem. There’s definitely a lot of analogies between the sensor package that exists on humans, and that’s what we’re most familiar with, and how that is similar to the challenges being faced when we’re trying to automatically position, ubiquitously position, more machines and vehicles or if it is humans at a much higher precision and accuracy than sort of the qualitative approach that the human brain and sensor package usually takes. So how do all of these different sensors work together to provide a highly accurate, highly precise indication of position and velocity and orientation as well?
SR: Thanks for listening to this episode of HxGN Radio. We’ve shared links in the episode description with more information about HxGN LIVE Global, or you can also visit hxgnlive.com/global. We hope you can join us in June.
