.

The Manufacturing, Engineering and Related Services ITAB (MERSITAB) was formed in 1995/96 when the National, Metal and Engineering Training Board and the Aerospace Industry training body were combined.

My background is that I was a motor mechanic. There's not a lot of grease under the fingernails now. I spent quite a few years in TAFE in NSW in various roles, and I've been with the ITAB for four years now.

Industry is committed to the training of workers for a variety of reasons. An arch capitalist would say, "we've got to have people to do the job to make the money and to pay the shareholders". Others would say, "it’s for the good of Australia and the public in general". ITABs represent industry in the public arena. We enjoy funding from ANTA, from DETYA (occasionally) and from industry to carry out our role.

There are several things to say regarding an industry perspective on literacy in training packages. One is the need and the demand by industry for people that have the general or generic skills (sometimes poorly titled 'soft skills') which enable people to work together, to interact and communicate with others, to process information and so on.

The Australian Industry Group that represents employers in the manufacturing industry, aerospace, building and construction industry, information technology (IT), printing, textile clothing and footwear, and so on, has a very broad-based membership. It commissioned a significant survey last year undertaken by the Allen Consulting Group.

There were several quite interesting outcomes from that survey. One strong outcome was that what employers valued most of all were non-technical skills, the things that made people useful and valuable employees. The fundamental skills of being able to communicate with others and work with others are very difficult to train people in or to instil. It's far better to employ people who already have them. Communication skills are probably the most valuable component of vocational skill. The technical skills can always be gained providing somebody has the general aptitude, interest, motivation and communication skills.

The survey said that we want people who can do all those sorts of things. We don't care about the technical side too much, because the workers of today and tomorrow will need the general skills to enable them to work in a changing work environment. Company X now typically has short-term contracts of supply and manufacture and the term of the contract might be three months. Hopefully they have something lined up for next year. No longer are there smokestack industries, producing the same product year after year. Employers want people who are adaptable, people who can shift from one task, process or job role to another. They want workers who can communicate and use work sheets and instruction sheets and produce reports at whatever level is required. Employers are obliged to have workers who are able to understand notices, hazchem sheets and all those sorts of things. Typical quality systems applied in Australia are heavily document- focused. Individual workers are becoming more autonomous and therefore they need to have greater self-reliance and they need the generic skills. Instead of somebody standing over them saying, "do this, do that", they actually need to self determine at their own level. They are working in teams (although these are a bit passé now) and in coordinated work groups. For their own own personal development, they'll also need the sorts of skills we're talking about.

Workplaces are becoming more and more culturally diverse, and this in itself creates a need for language, literacy, numeracy and communication skills. The key competencies are being addressed in the training packages in various ways and it's interesting to see that some years on now the key competencies still keep bubbling up to the top. Employers are still saying these are really important to us. The Mayer framework of key competencies is getting a bit old - it needs a bit of touching up in terms of detail because time has moved on - but fundamentally those competencies are still central required by employers.

MERSITAB has two Training packages at the moment, with another one coming along. We've got the Metal and Engineering Training Package which covers the manufacturing and engineering industry. We've got the aerospace one, called ‘Aeroskills’ We're also developing one for the recreational boating industry.

The Metal and Engineering Training Package was based on a set of competency standards that have been around in about four different forms and formats since the early 1990s. Their original production was for industrial classification purposes. The metal and engineering industry decided in the late 1980s that it would restructure the industry, and the way people were employed, by changing the federal industrial award. People would be paid for the skills they use at work, rather than the skills they possessed. The skills they use at work would be defined. Through a combination of curriculum modules and competency standards, they defined the sorts of things that people did at work. They then gave points of measurement. If the worker carried it out, then he or she had a rightful claim to be paid for doing that. A combination of units of competency gave a particular level in the industrial award and hence an amount of money. It is these competency standards that the Metal and Engineering Industry Training packages are built on.

From the industry perspective, the competency standards were primarily for industrial relations purposes and not for training. For that reason, every word, full stop, comma, colon, semi-colon etc has been pored over in painful detail: not by people like me, but by the industry parties that have participated in MERSITAB - employer organisations and unions. This is a bit of an apology as well. If you read one of the units of competency of the Metal and Engineering Training Package you'd say "Who wrote this? It's really hard. I don't understand this." The reason it's written like that, often obscure, is for an industrial purpose. Perhaps someone said, "we're not going to put up with this unless you put a comma in there, or an 'and/or' in there, or separate that out and make it another sentence, or another item in that list", or whatever it is. That's the environment the industry and ITAB people have to work within. Some ITABs and training packages don't have that same focus but quite a few do.

The Allen Consulting Group reported to the Australian Industry Group that employers are saying, "we value these ‘soft’ skills - literacy and numeracy are really important as well as being able to communicate, use IT and so on". However, when some employers were asked the hard question on literacy and numeracy, they said, "we want our workers to be literate and numerate and to be able to converse in the English language if required, but we're not going to pay for it to happen. If they can't do it, then that's tough. We think it's the role of government to do it." Ian Spicer, representing a large employers’ group, once said that literacy is really important but it's not the job of employers. It’s the school system’s problem. That perspective is still held by many employers. They value the benefits of a literate work force, but they're not prepared to support the development of literacy and numeracy skills in training.

So, if we specify levels of language, literacy and numeracy within our Training Package, that places an impost on the employer to ensure that that happens. "If I, the employer, am responsible for literacy, then we're not going to have a bar of it." So, industry will steer some away from that specification. What we've tried to do is to embed it. Hopefully, people can identify literacy competencies and say, "oh yes, I can understand what's needed here". But it's been very difficult.

The change in training packages away from a curriculum focus to an outcomes focus, and the deregulation of the training (so that registered training organisations could more or less do as they wish providing they produce the outcomes that we specify) has shifted the load back onto the RTOs to deliver the outcomes.

What I want to do is to talk about the role of assessment in training packages and the language, literacy and numeracy issues there. This is my personal view but it's certainly shared by some of the directors of our company and some of the organisations represented by MERSITAB.

Training packages have shifted the focus away from issues of learning as they only look at outcomes, and the performance of outcomes is determined through assessment. The other day, when I was starting to write these few words down, I had a look at the ALNARC brochure, at the vision. I thought "That's not too bad" because there was something in there that really grabbed me. The part that was interesting was the vision to contribute to the quality of practice, training, research and policy development in relation to the provision of adult literacy, numeracy, in community and industry based education and training. It's that industry-based education and training part that I'm interested in. It's about practice. I'm assuming that education and training includes assessment, and assessment includes the gathering of evidence and making a judgment on that.

The issue I want to raise centres on the distinction made in assessment processes between determining workplace competency and off-the-job competency. There is a distinction. My concern is the common identification of knowledge acquisition as being an off-the-job thing and the over reliance on written assessment practices. This affects the fairness of assessment as well as its flexibility and validity.

The term ‘competency’ has been in our common VET language for over 10 years now. We saw it initially in competency based training subjects broken up into modules. They gave a convenient method of breaking up large learning processes. However, these learning processes should lead to workplace competency. By tradition, developers of the modules sought to convert well established off-the-job learning processes or curriculum to a so-called competency model. I was involved in some of that work 10 years ago and I can assure you, and many of you would also know, that often the curriculum developers were miles away from what industry actually wanted. In defence, the industry didn't know what it wanted either, whichever industry it was. Often I’d go along to an enterprise and say, "okay, what sort of training do you want?" and the enterprise would say, "well you tell me".

The accepted view of workplace competency is the specification of knowledge and skill, and their application to a standard of performance required in the workplace now and in the future. Training packages incorporate workplace competencies. However many players in the Australian VET system continue to pursue the distinction between on-job and off-job ‘competency’.

Training package implementation plans from many of the registered training organisations provide a tacit or very weak link between institutionalised learning and workplace competency. What they do is certainly linked to the workplace but often it is not well linked. The majority of the vocational education and training effort by these organisations is directed towards completion of modules or similar curriculum pathways. This is for a training package which has workplace competency outcomes. For those learners who are being trained for the job they hold, as opposed to the majority who are training for their first or their next job, their performance against the modules is matched with often minimal workplace information. For those not employed in a job role related to their training, workplace performance is assessed by simulation.

Modules, courses and institutionalised delivery mechanisms have a valued place in vocational education and training in Australia. They can get large numbers of learners from point A to point B in a learning process, and do so efficiently. Institutions make some provision to support learners who are not sitting in the middle range of the ‘bell curve’. However, this form of provision frequently marginalises learners. The marginalisation can lead to situations where whole groups of learners who don't neatly fit the criteria of the disadvantaged target groups ("they aren't bad enough!") are left to deal with standardised or mainstream learning and assessment methods and materials. One of the critical things that we were always taught as teachers was to cater for individual differences. Yet we've got systems in place that actually don't allow for it. Where rigid criteria are applied to determine say literacy needs then there are often people who don't fit in the box but still have the need for support, and so the delivery and assessment mechanism can't cater for them.

Once decisions are made to use written testing methods assessment, an element of discrimination starts to creep in. The term ‘written testing methods’ should also include written response from candidates. Written assessments shouldn't be viewed as pen and paper testing only but written assessment tools are also used in online computer based resources and so on. Given the mainstreaming situation described above, for those who still have some need but don't fit the criteria to go into the ‘marginal’ group, what provision in assessment processes will be made for the outstanding language, literacy, numeracy, cultural and ability issues experienced by those people not swept up by the special provision process?

For those of you who have experienced an assessment, how do you feel about written tests compared with oral questioning and responses? Which was the most comfortable for the assessor? And, more importantly, which was the most comfortable for the candidate? Written assessment tools are efficent when big groups of people all do the same thing. It's quite efficient to develop a written assessment instrument that can be used by many candidates. It's not very easy to go through and mark them all but once the final assessment decisions have been made, the assessor has a record of the questions and responses for each candidate. By using the same assessment instrument, validity and reliability can be better assured. Flexibility can be enhanced by the design of the instrument but fairness often suffers. Fairness is often one of the key things that makes assessment either a good thing to do or something that many people are wary of.

Assessment through oral questioning and response is widespread throughout society. What would be the impact on the carriage of justice if our legal system relied solely on written questions and deposition? Think about that. Perry Mason would have been out of a job. I don't know how well he wrote but he certainly talked. Why is oral questioning so discredited when used for vocational education and training? When the national metal and engineering curriculum projects were under way in the late 80s and early 90s a standard module format was agreed to on a national basis. This agreed format included the use of phrases to describe standardised practices. For example, an assessment method used in most of the 1600 modules was for short-answer questions and practical exercises. Invariably, for ‘short-answer questions’ most people read 'written questions'.

If this singular interpretation of short-answer questions is applied to all assessment candidates, many will be disadvantaged. This disadvantage could equally apply to those undertaking a course and to those seeking recognition of skills already held. It particularly applies to disadvantaged candidates who've been away from formal education for some time. The use of jargon and obscure language in written assessment tools invariably raises the language and literacy requirements of the assessment beyond those required to perform the competency in question.

This gets back to the role of educationalists versus industry requirements in terms of looking after the needs of the student. From the industry ITAB point of view, the industry is correct. However, a balance is needed. Industry wants people that can perform at a particular level whereas we all know that individual learners need more if they are to progress. I think there is a reasonable compromise to be struck in education and training processes that tries to deal with each perspective.

In the guidelines for designing assessment materials in the Metal and Engineering Training Package, we say that, for assessments to be valid, language and literacy requirements during assessment should be no greater than the levels required to demonstrate competency in the unit being assessed. If you have a look at some of the assessment tools that are used, that's not the case. Many institutional assessors (teachers, lecturers and so on) do not feel comfortable when using only oral questioning techniques. They feel they have to have it in writing to assure quality. Many institutional managers don't feel confident that oral assessments will always be valid and reliable although they pay lip service to it. RTOs generally don’t promote oral assessment except in the case of special needs target groups. For these people, it is common to have the assistance of readers and writers during examinations and tests. Why can't others use similar techniques?

We are now in a process of transition from outcomes being described by curriculum-based courses to outcomes being described by workplace competencies. The change process is difficult for some, and many VET practitioners have opted to keep as much of the old as possible. This curriculum approach to workplace competency using the modules means that many learners will be offered training and assessment activities that are no longer relevant or appropriate. For example, most of the Metal and Engineering modules do not align very well with the competency standards. They were written at a different time for different purposes. There was a whole raft of them updated about 1996/7, in the updated modules. The content is better aligned, however, the old methodologies remain, including written assessments.

In the Metal and Engineering competency standards, we have varied our format from the common model so that we can include direct guidance for assessors. Against each performance criterion, against each element in the unit, we've included two statements. The assessor is to observe that so and so has happened and to confirm that - whatever the statement of competency is. The statements are designed to help assessors focus on the items that are critical to the demonstration of competency. ‘Observation’ can certainly include direct observation of behaviour but it can equally include consideration of evidence of behaviour such as supervisors’ reports, authenticated completed work and so on. The confirmation component is about the assessor being satisfied that the candidate has sufficient knowledge and the capacity to apply the knowledge in new and different situations and contexts. Typically the assessor would ask a range of 'what if' questions as their method of confirmation.

MERSITAB encourages the use of workplace assessors to carry out full competency assessments. This is encouraged through the assessment arrangements where a workplace assessor works in conjunction with an RTO teacher. Irrespective of the assessment method used, we expect that all assessors will record the details. Where oral techniques are used they need to plan and organise and keep notes in detail of what was done. That way, it should be still valid and reliable, and certainly a fair process.

What I want to leave you with is a challenge. And the challenge to ALNARC, or any of you, is about that vision: the vision of contributing to the quality of practice. What can ALNARC do to improve the quality of assessment practice? How can the use of non-written assessment techniques be increased? What can ALNARC do to influence policy- makers with regard to assessment? Those are the points I want to leave with you.

One last point. Under an ANTA sponsored program, we produced Making Sense, a resource for assessors dealing with language, literacy and numeracy issues. It gives examples of good and bad practice. It doesn't try and make assessors into literacy and numeracy experts but it says, "These are some of the ways you can overcome the difficulties and if you have a problem then this is how to find an expert".

Finally, have a look at the ANTA website: there's a section devoted to workplace communication resources. You might be interested to compare the different approaches to assessment taken by different industries.