First steps in translating human cognitive processes of cane pruning grapevines into AI rules for automated robotic pruning

. Cane pruning of grapevines is a skilled task for which, internationally, there is a dire shortage of human pruners. As part of a larger project developing an automated robotic pruner, we have used artiﬁcial intelligence (AI) algorithms to create an expert system for selecting new canes and cutting off unwanted canes. A domain and ontology has been created for AI, which reﬂects the expertise of expert human pruners. The ﬁrst step in the creation of an expert system was to generate virtual vines, which were then ‘pruned’ by human pruners and also by the expert system in its infancy. Here we examined the decisions of 12 human pruners, for consistency of decision, on 60 virtual vines. 96.7% of the 12 pruners agreed on at least one cane choice after which there was diminishing agreement on which further canes to select for laying. Our results indicate that techniques developed in computational intelligence can be used to co-ordinate and synthesise the expertise of human pruners into a best practice format. This paper describes ﬁrst steps in this knowledge elicitation process, and discusses the ﬁt between cane pruning expertise and the expertise that can be elicited using AI based expert system techniques.


Introduction
The development of an automated robotic pruner for cane pruning poses problems quite different than those of mechanised spur pruning.Cane pruning is more complex than spur pruning, involves greater understanding of vine growth, which canes to select, and takes longer per vine, thus the cost is greater and the available expertise more limited.This project links the fields of viticultural expertise and expert systems, since in order to capture both explicit and implicit cognitive processes of expert cane pruners for translation into Artificial Intelligence (AI), quantitative objective evaluation of the quality of cane pruning by both humans and by the robot will be needed.Here we describe how the cognitive processes of cane pruning might be analysed using methodologies of cognitive engineering.
Winter pruning is a major expense in a vineyard budget [1][2][3].It is the single most influential management technique in the vineyard year [2,4], as it is able to manipulate ripening progress, reduce labour requirements during the growing season [1][2][3] and target the yield demanded by the winemaker to achieve the style of wine perceived to address market demand [5,6].
Mechanical spur pruning is currently in use in the industry internationally.In spur pruning the cordons ("arms") of old wood are already in place, with spurs ("positions") from which generally two shoots have grown.One shoot is removed entirely and the remaining shoot is cut leaving only two buds, from which two new clusterbearing replacement shoots will emerge in the coming a Corresponding author: valerie.saxton@lincoln.ac.nz season.Selection of where and what to cut is generally considered undemanding intellectually.The process has been mechanised, although mostly in large lower quality vineyards.
Cane pruning is preferred in some cultivars for several reasons including increased bud fruitfulness, shoot uniformity, increased vigour and reduced costs of handpicking at harvest [1].Cane pruning involves selecting new canes from the head of the vine, complete renewal of the vine architecture, a process that may alter the vine's growth and productivity for a period of up to several years.For this reason poor cane pruning can be economically damaging, while at the same time finding enough skilled human pruners to complete the task in time is a world-wide problem.
In addition to unpleasant cold winter conditions working outside, human pruners suffer musculo-skeletal disorders and occupational overuse syndrome (OOS) frequently develops, even when powered secateurs (or ergonomically designed secateurs [4,7,8] are used.Even with careful workload management [9].There is considerable strain on the human body, especially wrists and hands, shoulders and backs [3,4], which even powered hand-held pruners do not always diminish [7].
With lack of human pruners due to all of the above, the case for developing robotic pruning is established.In addition, a link between pruning and the newly developed field of precision viticulture has been explored by authors who show that data collected during the growing season on yield and vegetative vigour, can be related back to winter pruning [10,11].So far any link between precision viticulture and pruning still relies on human pruners to do BIO Web of Conferences the actual pruning, but the potential to share data with an automated system is clear.
In spite of difficulties getting cane pruning done, there is still a great demand for this type of pruning.While the long running debate between spur and cane pruning to obtain an optimum yield and quality balance continues [5,[12][13][14], the case for cane pruning rests to some extent on cultivar, in that with some cultivars longer canes can capture the greater fruitfulness of buds farther along the cane.A second factor is that the quality of the resulting wine, especially in cooler climates, often attracts a premium price where the greater expertise of human cane pruners is worth the cost.However the skill required is greater, and the world-wide shortage of skilled workers is acute.
For automated robotic cane pruning to be successful, 3D vision must be used to underpin decisions about which canes to leave to achieve the desired growth pattern and number of viable shoots for next season.A hierarchy of decision making algorithms will be needed that must reflect the complex decision making of expert human pruners.
The human cognitive process of cane pruning has received little attention in the viticultural scientific literature.Balanced vines are essential for sustainable production [13,15,16] so knowing how the vine will respond to pruning (canopy density, shoot positioning, crop load) in the following and subsequent years is one element of the skills associated with cane pruning that denotes an expert [4].
The teaching and learning of cane pruning skill is not well structured in the New World industry.There are instruction/teaching files on the web, videos on You Tube, and textbooks.The basic principles are usually taught in technical courses and in practical workshops.Most cane pruners are given initial face to face instruction in the vineyard and some monitoring, and then continue to learn "on the job".There is, to our knowledge, no reported research on the actual skill of selecting canes for sustainable and optimal vine growth and wine quality.There is acknowledgement that experienced pruners are quick and efficient [8,14] and with vine piece-work can earn a living wage if they work fast.It is also evident that poor pruning can be devastating, to the point of vine decline, and that it will at the least take several years for vines to recover (S.Rohloff, 2012, pers comm).At the same time winter pruning is not a pleasant task; compared to other skilled occupations it is not well paid, so that it is difficult to find enough experienced pruners, and those who are experienced are often not out pruning but have moved into managerial roles.Accessing the skill of human cane pruning experts cannot be done remotely.Learning over many years "on the job" fits squarely into the category of implicit or tacit learning [17] which is difficult to convey explicitly, i.e., through education [18,19].Cognitive engineering research methods may return the implicit knowledge we need in explicit form to help design an expert system for the automated robotic pruner [20].
Similar to a method suggested by Boot and Ericsson [19] of designing simulated tasks to identify strategies and cognitive structures, a preliminary project was devised using computer-generated virtual vines [21].The results of this preliminary excursion into evaluation of the quality of robotic cane pruning are presented here.

Materials and methods
In the first instance an expert pruner on the research team ("domain expert", [22]) created an algorithm for cane selection, canes that would be left for laying down or for spurring.This algorithm was in a "pseudo-language" similar to basic, which were then programmed into the computer.With these as a base the computer designed a set of 100 vines with simplified skeletons [21].Each vine was different, each had a trunk, a head, two cordons representing 2 year old wood, and between 5 and 12 canes (last year's growth), most of which (but not all) arising from the head.On some vines some canes derived from farther along the cordons.Further detail was: black patches on the young canes to indicate breaks or damage, bull canes were a pale green, spurs that had been left last year were indicated, as were new canes that had grown from those spurs (see Fig. 1 for a sample vine).
The vines were to be pruned by selecting canes to leave for next year, canes to cut off at 2 buds as renewal spurs, and all remaining material to be cut off completely.
In our pilot study the robot AI pruner virtually 'pruned' 100 virtual vines, and so did 12 human pruners.The human pruners were volunteers who were employees of large vineyards in Marlborough and Waipara, New Zealand.In total, 12 pruners completed 100 vines by marking on paper in different coloured felt tip pens the canes they would select to lay down, and the canes they would select as renewal spurs.The computer applied the algorithm and also selected canes to lay and canes to spur.The computer's 01016-p.2preliminary pruning was evaluated by the human domain expert [21].
The 12 human pruners' "pruning" has now been analysed by collapsing the cane choices of the 12 pruners for each vine, to see how homogenous their choices were.Results for the success of the AI were analysed and published [21].This paper examines the homogeneity of the human pruners' decisions.
On each virtual vine the canes were numbered and a spreadsheet set up where the choice of each of 12 pruners was recorded for each cane on each of 60 vines.Percentages of choice were calculated and chi square applied to the results to calculate a p-value.

Results
Human pruners took between 35 and 115 minutes to complete the cognitive process of selecting canes on paper to leave on 100 virtual vines.
They were not instructed on how many canes to leave.About half the pruners pruned to four canes, and half to two canes plus two spurs.Some vines had three canes selected.Most four cane vines had no spurs left, some vines had only one spur selected.
Of the canes on each vine that were selected to be left either for laying or for spurring: 96.7% of the 12 pruners agreed on one cane choice 92.1% agreed on a second.83.6% agreed on a third.72.9% agreed on a fourth.(Six vines did not have a fourth cane selected).56% agreed on a fifth cane, though only 16 vines out of 60 had a fifth cane selected.
A chi-square on the first four values against 100% expected agreement indicated a p-value of 0.002.The rate of divergence on the first and second canes is about the same (3.3% vs. 4.6%).Analysis of residuals indicated that third and fourth choices were more different.

Discussion
Results indicated a high level of agreement on the first cane and on the second, but significant difference of opinion regarding subsequent canes.The final aim is to get 100% concurrence in pruning method so that AI can be programmed and the robot will prune to the satisfaction of all likely critics, though whether 100% agreement between experts in any field is ever likely is debatable [20,22].In any case the best possible repository of expert knowledge is needed for the robotic pruner project.
It is not necessary at this early stage of development for the robot to distinguish between four and two cane pruning.Both are covered by a selection of four canes that are best for leaving.The automated robot will select four canes to leave and cut off all remaining growth, A mechanical brush pulling machine (Klima TM or similar) will then come through removing all cut off material.The third stage, cutting canes to length, or spur, wrapping and tie down will still be performed by humans.Some of the differences in opinion might have originated in lack of instruction to the human pruners.We did not define what type of pruning our human pruners should do.This was in part to discover some of the disparity in pruning practice in the industry.We could have reduced options by asking all pruners to select four canes on all vines, or two canes and two spurs.But this would have distorted the pruners' options since some of the vines did not lend themselves to four canes or two spurs, and on these occasions sometimes only one spur was left, or only three canes selected.This reflects expert opinion which is that no spur is better than a poorly positioned spur, "mistakes breed more mistakes" (S.Rohloff, Pernod Ricard manager, pers.comm.2012).This project was preliminary and worked on the premise that pruners have their own way of doing things and it was at this stage more meaningful to find out what pruners were actually doing than to dictate how they should choose.For the major project planned the decisions will be made on real vines.There will still be vines that will be difficult to prune, where expert decisions may differ.Of the various reasons for experts to disagree would be the basic differences in target production for the next season.Pruning for high yield might mean leaving more buds or more canes, so already decisions will be different.The robot must be able to factor in such differences.
The fewer canes available for selection the more agreement there was between pruners.The number of canes available on any one vine for selection can be very variable, which might produce difficulties in decisionmaking, especially on a vigorous variety like Sauvignon Blanc which tends to have extra bull canes direct from the head.On the virtual vines bull canes were indicated in yellow-green but were rarely selected by humans even for spurs.This might indicate that pruners have in the past traditionally been instructed not to select bull canes even though they maybe in fact good for spurs.This would be symptomatic of simplified pruning instruction, where bull canes are not suitable for laying down for next year's cordon because internode length is likely to be longer than wanted, (and thus shoot spacing wider and fruit production less) and in addition these canes are usually brittle and may be only weakly attached to the vine head.But these are not factors that need to be included in spur selection, and in fact a vigorous bull cane might make a good spur that will throw two good shoots suitable for laying in the following season, and is likely to be well positioned on the head of the vine.
Combining the views of different experts usually involves conflicts [18] and this system did throw up some conflict in cane selection.The knowledge engineer must therefore attach weighting to different factors to allow synthesising of opinion in the expert system.This was done in our preliminary work [21] by the domain expert.
In order to compare the performance of the robot on the virtual vines with that of human pruners, ranking was applied by the domain expert [21].Under this ranking system the robotic system successfully pruned 30% of vines better than humans and 89% at least as well.When complex choices have to be made then a ranking of the importance of various factors enables a hierarchy of procedures to be determined, so that an end can be arrived at.Ranking may change according to the target 01016-p.3BIO Web of Conferences of the vineyard in grape production, since factors such as high yield and high quality (and others) can be mutually exclusive.These types of decisions can be structured in the software as decision centred design (DCD), which enables key decisions to be to be easily altered, instead of trying to encapsulate all decisions in the program [23].Human experts will help in the identification of such key decisions.We hope to elicit such differences by means of cognitive engineering techniques.
We hope to align cane pruning expertise with other fields that have used cognition techniques to successfully elicit tacit knowledge.Implicit or tacit knowledge, is procedural knowledge (knowing how) added to declarative knowledge (knowing that) to attain a higher level of competency than can be explicitly taught [18].Pruning is an activity where experts and experienced pruners have "expertise" which enables them to do the job faster and better than novices, even when the novices have been expertly instructed and follow the instructions given [18] and thus pruning fits the definition of implicit knowledge.In addition this expertise is not due solely to experience or "years on the job", but incorporates individual approaches such as more specific goals and analysis of the task or problem [24].The ability to look ahead, envisage results, predict the future is core to expertise [24,25], and is the single most important attribute of an expert pruner, who can envisage the future growth of the vine subsequent to his pruning choices.This lines up closely with chess experts, whose expertise was demonstrated to include evaluating the relative merits of different moves, evaluating the resulting position, systematically comparing consequences, discovering new moves that were superior.Parallel skills also included the ability to rapidly perceive the relevant structure, identify weaknesses, encode the end product, and generate potential moves by planning.Depth of planning increased with greater chess skill, increased time available for planning increased the quality of the moves, and increasingly the ability to encode and manipulate internal representations and plan the consequences [20] were all attributes of a chess player that are part of skilled cane pruning a vine to sustainable and optimal health and productivity.
Implicit or tacit knowledge cannot be explained, but can be observed in a practical context [17,18].Most pruning is explicitly instructed, but sadly it is a reality that the task is unpleasant enough for most workers not to do it enough to gain the expertise that makes them expert pruners.Those who could tend to take on management roles, or the task of instructing others to prune.With the hundreds of thousands of vines that need pruning, the lack of expert pruners is dire in New Zealand and internationally.In this project we hope to capture the expertise that cannot be explicitly transferred by using cognitive engineering techniques capturing knowledge through "stories" about situations, or ongoing commentary while working.Other techniques include self-questioning where the expert constructs the questions according to his own knowledge, uses metaphors to paraphrase situations [26].Case based reasoning (CBR) is a type of programming where past experience of human specialists (stories) is stored in a database for later retrieval when a similar case is encountered [25].
A further aim for this project is to develop an ontology [22,25], which is a set of specific terms that conveys the specific knowledge from experts to the knowledge engineer.The "jargon" of vine pruning fits with concepts in the area of semantic ontology.Terminology used by pruning experts will be recorded and used for the ontology of the domain.Some skills remain tacit simply because no one has asked the right question.Ambrosini and Bowman [26] suggest several techniques to code explicitly the implicit know-how, and some of these would be appropriate for our project.The terminology of cane pruning will be central to this process and will enable precise definitions for artificial intelligence.

Conclusion
Winter pruning of grapevines is an exacting, strenuous active activity that can be stressful on the body and hands and unpleasant in poor or cold weather.There have been various advances in viticultural technology, so that precision viticulture, which uses vigour or yield assessments to determine bud number to be left, could be invoked to drive pruning decisions such as node number or internode length.But the skill of cane selection still remains un-researched.Ergonomic research has developed semi-mechanical hand tools to reduce mechanical stress on hands and wrists.Automating this vineyard task and potentially linking it to precision viticulture will lead to higher quality pruning.In addition, defining the many steps will lead to a clearer understanding of the expertise involved in cane pruning, and the development of a code of best practice, all of which could contribute to a more sustainable pruning regime.

Figure 1 .
Figure 1.Sample of a virtual vine.Each vine was different.The vine has a vine head, last year's spurs, disease is noted as black sections of the cane, bull (or head) canes are shown as green.