Remote future

Over the last 25 years improved wireless communications, variable frequency drives, and better and cheaper colour screens, have shaped how operators control their cranes. The controller transmitters developed using those technologies, a belly box with proportional joysticks and a small colour screen, evolved rapidly, but may now be considered to have reached maturity, with only incremental improvements likely.

Over the last few years though, a new bundle of technologies has begun to enter both consumer markets, and some large industries. Virtual reality devices are now common in many homes. They are regularly used in some segments of the construction and engineering sectors in project planning. Artificial intelligence and machine learning drives both software services and physical equipment. Improved digital compression, and communication systems, now makes it possible to transfer massive amounts of data using a handheld mobile device. How might these innovations shape the future of crane control?

THE VIRTUAL CAB

One segment of the crane industry is already making increasing use of virtual reality. Operators of forestry cranes perform a skilled task, in difficult conditions. Working deep in the forest they use knuckleboom cranes to rapidly transfer cut logs onto their vehicles. Typically they work from a standing position next to the crane column. This gives them a clear view of the load, allowing them to work at pace, flicking their crane back and forth as they collect logs. But it can be a far from pleasant working environment, particularly in bad weather.

A few years ago Hiab launched a new way of operating these cranes. Rather than the operator standing behind the crane a camera system would be mounted in this position. The operator, wearing virtual reality goggles, could operate the crane from their driving cab using armrest-mounted joysticks.

Hiab now has HiVision systems out in the field, and offers examples of its use from Japan, Germany, and Sweden. As one equipment owner, Maik Ungefroren of Maik Ungefroren Transport und Baustoffe, in Germany, points out, the use of the system promises to make it easier to recruit new staff. He says, “I think that the advantages of HiVision are best represented by the safety and comfort of the operator, the increased payload, and above all the attractive work environment. This is important, because I believe that in order to attract young people and new employees, the work environment must become more appealing.”

Tadashi Hatakeyama of Furusato Mokuzai Logistics in Japan, makes a similar point, saying, “The big advantage of HiVision is being able to operate inside the cab in a calm way. In bad conditions, such as rain, snow or storms, we can work safely and improve our working environment.” Like Ungefroren, he believes this may help bring new people to the industry, and suggests this may increase the number of female operators.

OFFICE-BASED OPERATING

Hiab’s HiVision moves the operator from standing controls to the cab. While the vehicle cab may be more comfortable than any other position on a crane, an office is for many people an even more appealing working environment. But can controls be moved away from the crane entirely?

That’s certainly the case in the port crane industry. With high speed data transfer, it is now increasingly possible to work on cranes from far longer distances.

Liebherr offers two services that suggest how long distance communications may make crane installation and operation more efficient. Currently, port crane manufacturers like Liebherr must send service staff around the world to assist with and supervise crane installation. During the height of the pandemic, the company was due to deliver two harbour mobile cranes to Campana in Argentina. Normally, expert engineers from Rostock would fly out to the port, and supervise the crane’s assembly by local service technicians. This would have been extremely difficult, if not outright impossible, under the travel restrictions in place when the cranes arrived, in May 2020.

Instead, two of Liebherr Argentina’s service technicians, Alan Irazoqui and Cristian Bovino, would, for the first time, assemble the crane on their own. But they were not without support. Back in Rostock, service engineer Nils Liesner was able to follow their work using Liebherr’s Remote Service tool.

Ahead of work starting, customer service manager Christian Lubke reports, cameras were installed in the harbour and the Argentinian mechanics were equipped with mobile phones, whose high-quality cameras they could use to send Liesner close-up photos over the app.

Irazoqui and Bovino had a 60-page installation guide to work from. But the company was not confident this would be enough. Initially, they worked by sending still photos from their phone. But, once a robust internet connection was in place, Liesner could follow their work more closely. Each morning, online meetings were used to plan out the day’s tasks and to share his experience on the most efficient way to work, just as if he was on site with them.

Now the Covid crisis has eased, the company is likely to return to on site supervision of crane installation. But, when maintenance is required, customers and local service staff will be able to receive support, without waiting on a technician to fly from Rostock. The company is already using VR goggles for service support in its aerospace division, and these may also find a use for crane service.

In day-to-day operation, Liebherr’s Remote Operator Station allows for port crane operators to work from an office. The system looks rather like a standard office desk, with multiple built in displays, and control devices. It allows operators to control their crane, just as they would if they were working from the cab.

Late in 2021, Liebherr delivered four ship-to-shore cranes and 12 rubber-tyred gantries to the port of Duqm in Oman. Supplied without a cabin, the cranes will be operated utilising a combination of remote control and automation. Automation will take care of the majority of the cycle with operator intervention only required when operating below a predefined safe height.

The cranes at Duqm are networked together using a mesh system. Traditional communications networks take a hub-and-spoke approach: on a local network, all data is transferred through a router; on a wider scale, similar tools connect all of an ISPs network to the internet, and so on. Mesh networks are employed at home, on systems like Eero, Nest or Orbi. Each part of the network communicates back and forth with its neighbours, increasing the robustness and reach of the network. On a port, or a busy tower crane site, this approach reduces the risk of losing communications with the crane at a vital moment.

Similar systems are now being used away from ports, in industrial environments. Systems integrator TruTegra installed a system like this at an aluminium casthouse in a remote part of Canada. The plant is in continuous operation, with multiple bridge cranes used to transport ladles of molten aluminium and pour ingots for the rolling mill. Mobile vehicles are present in the area along with foot traffic.

The company considered two approaches to presenting a view of crane operations to the operator: VR goggles, and a projection dome. After trying both, the company decided that the vertigo some users experience when using goggles made the projection dome a better option.

TrutTegra used stereoscopic streaming and player software to manage multiple cameras and to produce the stitched image for both the 3D goggles and for the projection dome, before settling on the latter for the finished system. In addition, position data from the crane was transmitted back into the vision system to produce visual enhancements (lines and hash marks) that show the operator a projection of the hook onto the floor, as well as HMI (human-machine interface) and fault data.

The cameras on the crane needed to be protected from the extreme environment. Multiple vantage points had to be provided since the process of grappling the ladles and pouring the ingots requires extremely good awareness and control.

Multiple cameras mounted on the mobile crane created very large amounts of streaming data, which was managed by the streamer software. Custom driver software and application software was developed to create augmented reality (AR) features to aid the operator. A two metre spherical projection dome was constructed and equipped with three projectors to display the live streaming image with augmented reality for the operator. The operator can switch vantage points as needed to manipulate the crane and its load.

AUTOMATED LOAD PATHS

A key skill for tower crane operators is moving the load as swiftly as possible, selecting the best path from picking up the load to placing it. But could a machine do better than an experienced operator? Israeli automation firm Syracuse believes so.

While artificial intelligence and machine learning have been buzzwords in automation for many years, Syracuse takes a different, more traditional approach to automation, relying on the rules of physics to calculate the fastest load path, considering load sway and other factors. A human looking at the problem will normally choose the shortest path; an artificial intelligence, trained using human actions as a model, will make the same mistake.

Syracuse, which last year recruited former Manitowoc director of engineering of tower cranes Mathias Hess, uses physics-based robotic motion planning to select the best load path. The company generates a 3D model of the construction site and then identifies all vacant points through which the crane and load can move.

This results in millions of points, with every point representing a particular configuration of the crane and load. The computer then connects each of the points with all the others, to create a multitude of possible movements. It then removes invalid movements, where there is an obstruction. This process leaves the company with a roadmap of valid movements, as well as the value of each series of movements.

With this approach, and the ability to perform thousands of calculations simultaneously, Syracuse can perfectly model the complex relationship between crane movement and load sway and, it says, execute a 100% collision-free hyper-efficient load transfer.

The company is partnering with local tower crane firm Skyline, which has both Yongmao and ENG cranes in its fleet. The system was recently deployed on a flat-top tower crane, on a construction site comprising a large 18-story residential building, in downtown Tel-Aviv. This site joins two additional Syracuse-enhanced hammerhead tower cranes, featuring both operator-assistance and autopilot functionality.

LOAD CONTROL WITHOUT TAG LINES

On many job sites, one of the key safety roles, controlling the orientation of a load, is often left to some of the most junior lifting crew, slingers and signallers. Using a tag line, these personnel must ensure the load is held steady. If the load is dropped, they are left standing where it falls; if it is caught by the wind, the rope they hold may be suddenly pulled away.

Over recent years, three companies — RoboRigger and Verton in Australia, and Vita Inclinata in the USA — have developed systems that hang below the hook, automatically orienting load.

Earlier this year, Sumitomo Mitsui Construction Company (SMCC) in Japan used its own construction information system, in combination with RoboRigger’s load orientation system, fitted on an IHI tower crane, to test an autonomous lifting system for precast concrete elements.

The entire site is mapped in a building information management (BIM) system, and each element is tracked upon delivery using RFID tags and GNSS satellite positioning.

These data are used to calculate the best path for the load. In the current prototype Tower Crane Operation Support System, an operator confirms the path calculated by the system for each load. As the pre-cast element is lifted automatically across the site, the RoboRigger orientates and adjusts its position.

SMCC says the system will help reduce the impact of a shortage of skilled operators, and will be safer and easier than manual operation.

RoboRigger, and its Australian counterpart Verton, both use gyroscopes to orient the load. US-based Vita Inclinata’s Load Navigator uses small, high torque, electrically powered thrusters on either side of the system to adjust its orientation.

The Load Navigator was recently tested out on a tower crane erection by Creative Lifting Services (CLS) in Boulder, Colorado. When installing tower crane sections, rigging crews on the tower crane must normally communicate with their colleagues on the ground, using tag line to position the elements. Vita Inclinata’s Load Navigator instead uses two sets of remote controls with a ‘handshake’ system to pass control between the team. This eliminates the need for staff on the ground to stand below the load, while ensuring that the riggers closest to the installation point can precisely position the load.

When using a mobile crane to lift long tower crane elements, there is always a risk of the load spinning or being caught by a gust of wind, and striking the telescopic boom. On this first job, CLS kept tag lines in place as a back-up protection against this risk. However, with the Load Navigator able to maintain the correct orientation of loads of up to 40t in wind speeds of up to 30mph, these were not needed.

AUTONOMOUS ON-SITE TRAVEL

On a large site, moving mobile equipment can be time-consuming. But it is not the sort of task that really challenges an equipment operator. What if equipment could move autonomously, without an operator, freeing them, and their skills, for where they are most valuable?

That’s the approach being taken by SafeAI. The company recently partnered with Siemens and Obayashi to develop prototypes of its retrofitted autonomous solution in Cupertino, California.

SafeAI explains that its system allows heavy equipment owners to convert aftermarket vehicles and fleets through its retrofit autonomous vehicle hardware kit and industry-specific software, regardless of vehicle type or manufacturer, while delivering significant gains through increased productivity, safety and cost savings.

On the trial site, the companies have, since October 2021, successfully tested and deployed an articulated autonomous dump truck to complete over 580 load-haul- dump cycles. Earlier this year, a fleet of construction trucks ranging from 45 to 65 tons and operated by Obayashi Corporation was identified to be retrofitted for autonomy and zero emission. In May 2022, the companies began retrofitting the first 45 ton vehicle, which is expected to be ready by the end of 2022.

The collaboration will continue scaling across the entire fleet over a three-year period.

THINKING ABOUT THE FUTURE

So, how might these technologies come together on cranes used in the construction industry? It seems unlikely that autonomous operation will be of immediate value to crane hire fleets taking on taxi work and other short term jobs. Site conditions will be too varied, and while the ability to drive cranes long distances to site without an operator could cut some costs, this will be constrained by risks inherent in mixing human- and machine-operated vehicles on public roads.

But on large sites, tower crane and crawler crane owners may see some more immediate opportunities. In many ways, multiple tower cranes working on a big project are similar to port cranes. Their positions relative to each other are generally fixed, or follow rails, making it easier to plan out all their possible movements and interactions. Tower crane positions on a large site are designed to offer efficient coverage across the site. Just as much as a well deployed fleet can ensure a load can be lifted from anywhere, so tower cranes fitted with mesh networking could provide robust data coverage, for themselves and for other networked equipment.

On any crane, many operators spend much of their time in the cab idle. By using an autonomous system, like those being developed by Syracuse and SMCC, idling time could be further reduced, with the operator only needing to directly supervise the more challenging lifts. With each crane on site only requiring occasional operator intervention, one operator could be assigned to multiple cranes. That could allow them to work from a desk, using something like Hiab’s HiVision, Liebherr’s remote operating system, or TruTegra’s virtual reality projection dome.

Perhaps initially, that would require them to work from a purpose-designed site office. But we have seen from Liebherr’s experience supervising crane installation in Argentina from an office in Rostock, that global-scale networking is now sufficient to allow for reliable video and data transfer. Perhaps in the future, experience crane operators and service technicians will be able to work from their offices, or even their homes, supervising the erection of cranes by local riggers, and operating multiple cranes from anywhere in the world that suits them.

This might be of particular interest to those small countries and city-states that wish to build world-beating skyscrapers, but lack a local workforce and do not, perhaps, offer the warmest of welcomes to foreign workers. Rather than a crane operator migrating from India to the Middle East, and putting up with the restrictions they might face there, could they not work in an office in Bangalore, keeping up their normal life and social connections, while saving the client money?

Another potential starting point for autonomous lifting might be on large crawler crane sites. On a large wind farm, operators will spend much of their time moving crawlers from one hard stand to the next. With a system like SafeAI’s, this part of the job could perhaps be performed autonomously, with operators working between multiple cranes, once they had arrived at their location and been set up.