Sustainability assessment tool for road transport asset management practice

by Prageeth Gunarathna and Rayya Hassan

In the last two decades, road transport agencies have been facing a challenging task to balance the provision of efficient services with limited resources, while at the same time addressing road transport-related negative impacts on sustainable development.

Road transportation asset management (RTAM) plays a major role in facilitating the adoption of sustainable services while responding to challenging environments.

These challenges include both influences from external factors, such as global climate change and resource shortages, and internal influences within an organisation, such as financial limitations, lack of leadership and lack of professional staff.

Many road agencies have already responded to these challenges by incorporating sustainability best practices into their asset management process, and some have developed measures and targets to assess their sustainability performance at different levels of the asset management process.

There is a considerable number of sustainability assessment tools that have been developed by different organisations around the world, but in all only part of the asset management process is considered.

The main aim of this research study is the development of a tool to help road agencies to assess and monitor their performance in addressing sustainability principles at each level of the asset management process.

The principles considered include all those relevant environmental, social, economic, corporate, financial and technological sustainability.

Integrating sustainability into RTAM process

Sustainability principles need to be combined with key principles and attributes of asset management for better sustainability performance.

Presented in this section is the systematic approach adopted to integrate sustainability principles into the RTAM process and develop process-based performance measures for assessing sustainability performance.

This started with adopting a definition for asset management that promotes a holistic view of all the system’s assets.

The definitions of transport asset management adopted by different road agencies vary but the majority are limited in scope in that they are focused on life cycle management of the physical infrastructures (Austroads 2009; Cambridge Systematics Inc. et al. 2009; IPWEA 2006; OECD 2001; TAC 1999).

However, none of them explicitly mentions the different types of assets that play a vital role in managing the physical assets in road transportation.

In this study, the definition of the International Standards Organisation (ISO) ISO-55000 has been adopted as it is designed to apply to any asset type: “Asset management is a coordinated activity of an organisation to realise value from assets” (BSI 2014). Target asset categories are not defined within this definition and need to be decided by the organisation and stakeholders.

In this study, a holistic approach has been adopted when identifying relevant asset categories in consultation with relevant road agencies. To integrate principles of sustainability into the RTAM process, the following steps were followed:

  1. All processes and relevant tasks of RTAM practice were identified. To achieve this, Austroads’ integrated asset management framework (Austroads 2009) was used as a guide. It comprises three main parts including strategic planning, asset management actions, and performance feedback stages which are subdivided into seven phases. They include defining objectives (phase 1), forming asset strategies (phase 2), developing investment program (phase 3), identifying asset requirements (phase 4), implementing work program (phase 5), auditing (phase 6) and reviewing (Phase 7). Austroads’ reports (Austroads 2002, 2009) were used to identify all processes and tasks involved in these phases and their interactions.
  2. The risks associated with each task due to internal or external drivers were identified. Possible internal drivers (e.g. knowledge gap, resistance to change) and external drivers (e.g. climate change, legislation requirements) and associated risks that may influence the different processes/tasks were identified.
  3. For each identified risk, the relevant sustainability dimension was established and best sustainable practices were identified. Since the definition of asset management adopted in this study promotes a holistic view of all the system’s assets, the sustainability of RTAM performance can be considered multi-dimensional. This is further confirmed by the risk assessment process. Six sustainability dimensions are identified and include environmental, social, economic, financial, corporate and technological. It is important to have a working definition of sustainability, since it allows an organisation to identify key factors that should be invested in to achieve sustainable goals and the types of performance measures that can be used to monitor progress (DTMR 2012; Pei et al. 2010). The definitions adopted herein for these dimensions are as follows:
    • Environmental sustainability: This involves protecting natural resources, minimising waste and air pollution, and maintaining a variety of species and habitat through the RTAM process. It includes minimising resource consumption by improving material re-use and recycling, and by efficient usage; limiting air and noise pollution; minimising waste accumulation; and reducing the impact of transportation activities on the ecological system.
    • Social sustainability: This is the ability of communities to provide safe and healthy environments, while providing equitable and affordable services for their development. It includes providing a safe, secure and healthy environment; maintaining equitable opportunities within different social levels, and providing affordable transport services; and increasing community development by satisfying the basic accessibility needs of a society and its individuals.
    • Economic sustainability: This is concerned with the productivity of the asset systems for efficient services, service improvement and economic development. It includes building a transport system that is resilient in the face of climate change risks; providing efficient services; improving local development; and providing an economically feasible and affordable transport system.
    • Corporate sustainability: This is concerned with stakeholder satisfaction and the implementation of a sustainable culture within the organisation. It includes staff motivation and improvement of the target workforce within the organisation; valuing and developing the competence and capabilities of staff; enhancing leadership and stakeholder relationships; and implementing sustainable practices and regularly assessing cultures within the organisation.
    • Financial sustainability: This is focused on investing in long- and short-term financial needs to achieve target asset performance. It includes forecasting different financial needs and reserve funding; maximising value for money for all stakeholders; minimising unexpected financial shocks due to disaster situations; and forecasting future funding needs to replace and renew the assets as needed.
    • Technological sustainability: This is focused on advancing technology in the asset management system and minimising the use of obsolete technology. It includes partnering with client and key stakeholders to create or find innovative products and processes for sustainable transport solutions; and improving the technological capabilities of an organisation’s assets.

Framework for developing process-based sustainability performance measures

There is no standard framework for evaluating progress towards sustainability. The approach and methodology proposed herein for developing a framework for sustainable performance measurement for RTAM combines concepts from three frameworks.

They include the theme-based/ impact-based framework described in Hart (2006), Jeon and Amekudzi (2005) and Nichols, Garrick and Atkinson-Palombo (2009); the goal-oriented framework described in Nichols et al. (2009); and the influence-based framework described in Jeon and Amekudzi (2005) and Pei et al. (2010).

The conceptual framework proposed here in is presented in Figure 1. The main challenge of this study is to define a set of performance measures for the different phases of the asset management cycle.

To simplify the application of the proposed framework to Austroads’ asset management cycle and link the sustainability objectives to the different activities in the different phases, the cycle is represented by four focus areas.

The focus areas and relevant phases are: planning (phases 1 and 2); programming (Phases 3 & 4); implementation (phase 5); and operation and performance feedback (Phases 6 & 7). The framework in Figure 1 outlines the top down approach for developing sustainability performance measures and include the following steps.

  1. Define sustainability dimensions: the theme-based framework was used in defining the multidimensional sustainability concept. They include economic (EC), social (SO), environmental (EN), corporate (CO), financial (FI) and technological (TE) dimensions.
  2. Define sustainability goals: a goal-oriented framework was used to interconnect sustainability dimensions with relevant sustainability goals. The goals should be achievable and well balanced and need to cover the basic principles of sustainability and asset management. A comprehensive literature review resulted in identifying a set of 12 generic goals for transportation agencies to address the principles of sustainability in asset management. Table 1 presents the goals relevant to each of the sustainability dimensions which are common for all four focus areas.
  3. Define sustainability objectives: the goals were further deconstructed into achievable objectives that must be associated with the organisation’s strategic plan. The objectives are more specific than the goals and lay the foundation for links with the performance measures. The proposed 25 objectives shown in Table 1 are common for all focus areas of the asset management cycle.
  4. Define process assessment statements (performance measures) for different focus areas: Each objective was linked by process assessment statements for the different focus areas of asset management. An influence-based framework was used as the foundation for defining process assessment statements at different levels of the cycle by using examples of best sustainable practices. Process assessment is subjective approach and has been adopted by many of the sustainability assessment evaluation tools used in practice (ISCA 2013; Greenroads 2011; FHWA 2015; NYSDOT 2011; ISI 2012).

A total of 179 assessment statements were developed to evaluate an organisation’s processes relevant to each focus area, considering the 25 objectives. Table 2 shows an example of process assessment statements for objectives relevant to environmental sustainability. The first column of the table shows the target objective (using relevant code), the second column defines the process assessment statement and the third presents the relevant focus area.

Developing sustainability composite index using process assessment measures

The hierarchical structure for developing sustainability composite index (SCIprocess) using process assessment measures is shown in Figure 2. The approach used for determining the scores for agency’s practices is through application of the AHP (Austroads 2007) and consists of the five steps below. The weighting and scoring processes are important parts of the subjective assessment process; their details are in Gunarathna (2015).

  1. Develop weights for sustainability dimensions using AHP pairwise comparison matrix.
  2. Develop weights for sustainability goals using AHP pairwise comparison matrix.
  3. Develop weights for sustainability objectives using AHP pairwise comparison matrix.
  4. Rate/score the agency’s sustainability performances in each focus area of the cycle.
  5. Develop the composite index by combining all scores.

The hierarchy for developing sustainability composite index is presented in Figure 3 which combines the indices of the six dimensions.

The environmental dimension is used here in a demonstration example for developing a dimension-based composite index, and the same procedure can be applied to developing these for the five other sustainability indices. The steps for calculating the Environmental Sustainability Index (ENSIProcess) are described next.

The definitions of the different parameters used in the figure and demonstration are as follows:

WEN = Weight of EN;

ENSIProcess = Environmental sustainability index;

SCEN1 = Obtained score for goal EN1;

WEN11 = Weight of objective EN11;

SCEN11 = Obtained score for Objective EN11;

Si = Score for sustainability statement i; Sp = Statements related to planning level;

Sm = Statements related to programming level;

Se = Statements related to implementation level;

So = Statements related to operation and feedback level;

a = Number of statements for planning level;

b = Number of statements for programming level;

c = Number of statements for implementation level;

d = Number of statements for operation and feedback level.

Score performance statements for each sustainability objective

In each of the four focus areas, performance is assessed using the subjective rating. This consists of a number of statements representing best practices that are related to each objective relevant to each goal of the sustainability dimension being considered.

These statements are rated according to the agency’s sustainability performance using a five-point rating scale adopted from Liberatore (1987).

Shown in Table 3 are the priority weights of these five points that were determined by using AHP pairwise comparison matrices and their relative level of achievement (Gunarathna 2015).

The agency’s performance can be rated as O, G, A, F or P, depending on the level of achievement compared to the targets set out in their relevant strategy.

The latter are based on target best sustainability practices. The performance of each focus area is scored according to how well the agency has performed in achieving the sustainability objectives (e.g. EN11) that are related to the sustainability goal (e.g. EN1).

Sum scores for each focus area and normalise total scores by number of statements

Objective EN11, for example is assessed by a number of statements referred to as a, b, c and d for the planning, programming, implementation and operation and feedback focus areas, respectively.

These statements of relevant best practices are used to rate the agency’s performance using the five-point scale. The relative importance (weight) of each statement is considered to be equal when assessing each focus area.

The total score for each focus area can be calculated by using Equations 1, 2, 3, and 4. As the number of statements is different for the four focus areas, the total score of each area is divided by the relevant number of statements to normalise the scores.

This approach helps in identifying the focus area (s) in which the system is not performing. This process described is applied to all objectives relevant to the dimension.

Determine the score for each objective

This is done by adding the weighted scores of all focus areas, assuming all four areas are equally important and their relative weights are considered to be the same (0.25). Therefore, the total score for the EN11 objective can be calculated using Equation 5.

Determine scores for each goal

This is achieved by adding the weighted scores of all relevant objectives. For example, scores of all environmental sustainability objectives i.e. SCEN11, SCEN12 and SCEN13 can be calculated then these three values are combined using Equation 6 to obtain the score for goal EN1.

The AHP pairwise comparison technique is used to assign weights (e.g. WEN11, WEN12 and WEN13) for each objective based on their importance in achieving the goal.

Determine scores for sustainability indices

This is achieved by adding the weighted scores of all relevant sustainability goals. For example, the environmental sustainability index ENSI(Process) can be calculated by combining the weighted scores of all relevant goals using Equation 7.

The AHP pairwise comparison technique is used to assign weights (WEN1, WEN2, WEN3 and WEN4) for each of the goals by considering their importance in achieving the sustainability dimension.

Determine sustainability composite index

This is achieved by adding the weighted scores of the indices for the six dimensions.

The process in step 5 is repeated to calculate the indices for six sustainability dimensions and their relative weights are determined using AHP pairwise comparison considering their importance in achieving overall sustainability in RTAM practice.

Then these indices can be combined using Equation 8 to obtain a final score for the SCIprocess.

where:

ENSI(Process) = Environmental sustainability index;

ECSI(Process) = Economic sustainability index;

SOSI(Process) = Social sustainability index;

COSI(Process) = Corporate sustainability index;

FISI(Process) = Financial sustainability index;

TESI(Process) = Technological sustainability index;

WEN = Weight of environment dimension;

WEC = Weight of economic dimension;

WSO = Weight of social dimension;

WCO = Weight of corporate dimension;

Tool for assessing sustainability of process-based RTAM practice

An Excel-based tool has been developed based on the framework and processes described in the previous sections to help asset managers in assessing the sustainability of processes they are using in managing their assets and identify areas needing improvements.

It is called Road Transport Management Sustainability Assessment Tool (RAMSAT). In RAMSAT, the process for determining the composite index consists of scoring the 179 process assessment statements or performance measures relevant to the objectives applicable to each of the four focus areas.

It also involves entering judgements in 19 AHP pairwise comparison matrices for assigning weights to objectives (12 matrixes), goals (6 matrixes) and sustainability dimensions (1 matrix).

To improve the accuracy and validity of the ranking process, each matrix is checked for consistency of judgements by the tool, automatically.

This allows the user to identify the problem and improve the consistency of the pairwise comparison matrix to ensure a reliable sustainability index.

According to the assessor’s input, the RAMSAT automatically calculates the sustainability performance of an agency in achieving each objective and goal relevant to the six sustainability dimensions. Performance results are summarised in tables and graphs.

The tool includes a user manual with instructions and examples on how to do the scoring and weighting required in the different steps.

Figure 4 shows the menu page (Figure 4a) of RAMSAT and a sample output (Figure 4b). The user needs to complete the following steps in order to evaluate sustainability performance of the road agency.

  1. Assign weights for sustainability dimensions.
  2. Assign weights for sustainability goals and sustainability objectives.
  3. Assess the agency’s sustainability performance for each focus area of the asset management cycle.

After all necessary weighting and scoring processes are completed, RAMSAT calculates the sustainability performance of the agency and generates different outputs. The results can be accessed through the ‘process assessment composite index’ link provided on the menu page. The outputs from the tool include the following:

  1. Overall sustainability performance of the agency: an example is provided in Figure 4 showing that the road agency has scored 59% for the overall sustainability performance (SCI(Process)). The individual scores of the six sustainability indices (ENSI(Process), SOSI(Process), ECSI(Process), COSI(Process), FISI(Process), and TESI(Process)) are represented by the hexagonal chart. According to this example, the road agency needs to improve their performance related to the environmental, social and economic areas. The environmental index has the lowest score (<40%) among the six indices in this example.
  2. Three types of graphs are also generated by the RAMSAT for each sustainability dimension as indicated below.
    • Goal-wise performance for each dimension;
    • Objective-wise performance for each goal;
    • Objective-wise performance versus the four focus areas.

Therefore, the environmental dimension (EN) can be further explored to identify where the agency is poorly performing i.e. in achieving which goal and /or objective and relevant focus area.

Figure 5(a) shows the goal-wise performance of a road agency in achieving the goals of environmental sustainability which indicates that all four goals obtained low scores.

These scores can be further explained using the objective-wise performance chart in Figure 5(b). This chart can be used to identify the objectives where the agency is not performing well (e.g. objectives EN11 and EN13 in this example).

Figure 6 shows the objective-wise scores for each focus area, which can be used to identify the poorly performing focus area(s) in addressing the relevant objectives.

For example; objective EN13 performs poorly in programming, implementation and operation and feedback areas compared to the planning area.

Some of the objectives are not relevant to all four focus areas and these are scored for the relevant areas only. For example, EN12 objective is only relevant to the planning and operation and feedback areas focus areas.

RAMSAT can be used as a decision support tool at different levels of the asset management process to improve sustainability performance. Further, the progress of a road agency’s towards achieving a sustainable practice can be assessed on a yearly basis or over a defined time frame.

Conclusion

This paper describes the development process and functionalities of a process-based sustainability assessment tool for road transport asset management practice, referred to as RAMSAT.

Also, it briefly outlines the selection and development of sustainability dimensions, goals, objectives and process assessment statements which are the core aspects of the assessment process.

The development process adopts a holistic view of RTAM practice and includes six defined dimensions of sustainability to address emerging challenges. RAMSAT is a user-friendly tool that can be used to assess sustainability performance of any road agency.

Asset managers act as the assessors in the evaluation process to assign weights for the dimensions, goals and objectives and assign scores for each sustainability assessment statement. It helps asset managers to assess the sustainable performance of their practices in addition to identifying what and at which level(s) of the cycle changes need to be made for further improvement.

This paper included the example results obtained from the first RAMSAT testing process and testing is still continuing to improve validity of the tool according to comments from other road agencies before adoption.

This paper is focused on improving an organisation’s sustainability performance at each level of the asset management with limited focus on comparing sustainability performance of different road agencies. Each road agency has different strategies, policies and targets.

Further development of RAMSAT would be required to allow such comparison and a set of guidelines need to be developed to ensure fairness of the evaluation process for comparison purpose. In addition, the tool can be further improved by providing the facility to ignore the redundant sustainability statements, according to the agency’s policy/strategies in the evaluation process.

Appendix

Figure 1. Framework for developing process-based sustainability performance measures.

Figure 1. Framework for developing process-based sustainability performance measures.

Figure 2. Approach for developing sustainability composite index using process assessment measures.

Figure 2. Approach for developing sustainability composite index using process assessment measures.

SCI process development hierarchy.

SCI process development hierarchy.

Figure 4. Menu page and visualising sustainability performance of the road agency. WFI = Weight of financial dimension; WTE = Weight of technological dimension.

Figure 4. Menu page and visualising sustainability performance of the road agency. WFI = Weight of financial dimension; WTE = Weight of technological dimension.

figure-4b

Figure 5. Graph of scored values by environmental goals and objectives.

Figure 5. Graph of scored values by environmental goals and objectives.

Figure 6. Graph for scored values by environmental objectives at four focus areas.

Figure 6. Graph for scored values by environmental objectives at four focus areas.

Table 1. Goals and objective relevant for the six RTAM sustainability dimensions.

 

Dimensions Goal Objective Code
Environmental Conserve energy and Minimise materials and water consumption EN11
(EN) natural resources (EN1)
Maximise the use of land EN12
Improve energy efficiency EN13
Minimise emissions and Reduce air pollution EN21
noise pollution (EN2)
Reduce noise pollution EN22
Minimise waste Ensure waste management EN31
accumulation (EN3)
Minimise impact on Ensure proper functioning of eco‑system EN41
ecological systems (EN4)
Social (SO) Enhance safety, security Improve safety and security requirements SO11
and public health(SO1)
Improve public health conditions SO12
Improve equality and Improve participation in decision‑making SO21
provide affordable service
Equal services for all road users including low SO22
(SO2)
income groups and disabled people
preserve natural and cultural heritage value SO23
Economic (EC) Maximise economic Improve network systems efficiency and EC11
productivity by improving reduce congestion.
asset efficiency (EC1)
Improve resilience of transport infrastructure EC12
for climate change risk
Ensure economic Local economic development EC21
development (EC2)
Balance spatial planning EC22
Economic affordability EC23
Corporate Improve quality of life Improve staff motivation and improve target CO11
(CO) (CO1) workforce within organisation
Improve sustainability Sustainable purchasing and procurement CO21
culture within
Improve office-based sustainable CO22
organisation (CO2)
performance
Financial (FI) Improve organisation’s Ensure financial availability FI11
financial affordability
Maximise value for money FI12
(FI1)
Minimise financial risk in disaster situations FI13
Technological Ensure technological Improve research and development TE11
(TE) advancement (TE1) opportunities
Improve technical capabilities TE12


Table 2. List of process assessment statements for environmental dimension.

Objective Sustainability assessment statements Focus level
EN11 The road agency adopts policies and standard specifications Planning
for improving and prioritising material reuse and recycled
material use for transportation infrastructure construction
and maintenance activities.
EN11 The road agency’s policies include prioritising of Programming
pavement preservation effort or refurbishment of existing
infrastructure considering comparison of design alternatives
based on life cycle cost analysis and engineering judgement.
EN11 The road agency enforces use of recycled material for Implementation
infrastructure projects according to the specification
requirement and performs regular checks to verify quality
and quantity of the recycled material use in construction and
maintenance activities.
EN11 The road agency regularly (and quantitatively) demonstrates Operation and feedback
use of material for infrastructure projects and their progress
towards the defined environmental sustainability goals.


Table 3. Possible rating score for S
i.

Possible rating Priority weights (Si) Level of achievement
Outstanding (O) 1.00 ≈ 100% 60-100 % of target
Good (G) 0.59 ≈ 60% 30-60 % of target
Average (A) 0.27 ≈ 30% 20-30 % target
Fair (F) 0.13 ≈ 20% 10-20 % target
Poor (P) 0.07 ≈ 10% 0-10% target
0 Comments

Leave a reply

Your email address will not be published. Required fields are marked *

*

CONTACT US

We're not around right now. But you can send us an email and we'll get back to you, asap.

Sending

©2019 Energymagazine. All rights reserved

Log in with your credentials

Forgot your details?