The Awards Evaluation Criteria

The award is given to the institution and IT partner who has garnered the highest aggregate score from the General Criteria and Award-specific scorecards. The General Criteria scorecard measures the project management side of the implementation, and the Award Specific Criteria scorecard measures the category specific metrics and features of the evaluated solution. The scorecard is used by the researchers and Council of Advisers for Technology & Operations comprised of bankers, independent resource persons and academic professors.

The research team evaluates the submissions and conduct their own research in order to be able to complete the Scorecard for each of the elements listed below. The scorecard for the shortlisted submissions are published as part of our repository of research after the awards are announced.

I. General Criteria

1. Strategy Alignment
  1. Does the project/system integrate/align with organization business and IT strategy
  2. Does the project/system align and support business bureau goals
  3. Has the decision team taken consideration of the current economic, regulatory and technological factors that impact on the project/system
  4. Do all the stakeholders, direct human resources involved understand the ultimate goals of implementing this project/system
  5. Have the purpose and direction of implementing such project/system been communicated between business bureau and IT technical teams
  6. Are there any approval procedure/policy to govern the project/program
2. Delivery of the overall articulated business value
  1. Does the project/system streamline the specific business process
  2. Are there any procedures to govern the cost benefit analysis process
  3. Are there any methodology/tools are used to evaluate the cost and benefit
  4. Are there any on-going assessments in place to assure value delivery
  5. Are there any post project value analysis
3. Project management
  1. Did the project have a compelling business case
  2. Was the project charter completed and signed by all relevant stakeholders
  3. Were the outcomes, deliverables, benefits, assumptions and constraints well documented
4. Project Execution
  1. Were there project managers from both sides whose duties were clearly defined
  2. Was there a process in which they would measure performance and communicate issues that arose during the implementation
5. Vendor Strategy/Evaluation
  1. Was the vendor strategy aligned to the organisation
  2. Was the vendor's core competence identified
  3. Was there a selection process followed to select the vendor
6. Contract Development Management
  1. Were there performance drivers of vendors that were identified
  2. Were there clearly defined and robust SLAs in the agreement
  3. Was the relationship between the bank and vendor well maintained
  4. Was the transition process implemented smoothly
  5. Was there sufficient documentation provided by the vendors during the implementation
7. Clear Project Management Guidelines
  1. Did the project have too many scope creeps
  2. Was the technology defined in scope?
  3. Was there any process followed to control the scope changes during the project
8. Cost Management
  1. Was there a proper assessment of the cost of the project
  2. Were there tools used for cost estimation and analysis
  3. Did the stake holders accept the total cost spend on the project
  4. Did the project have enough budgets throughout its life cycle
  5. Was there any tool used to record, monitor and approve the cost of the project
  6. Did the project budget change frequently
9. Meeting Quantifiable Targets
  1. Is the defined schedule acceptable given all project tasks
  2. Did the project have enough buffer to cover the module/milestones delay
  3. Did the actual delivery vary from the scheduled delivery too much
  4. What were the tools used to monitor/track the timeliness of the project
10. Meeting Goals
  1. Did the project involve regular meetings with stakeholders on project updates
  2. Did the project have regular testing with proper documentation
11. Risk Mitigation
  1. How were the risks identified, managed, reported and resolved during the implementation
  2. Was there a process of escalation when issues and risks became too difficult to manage

 

II. Award Specific Criteria

1. Best Core Banking Implementation
  1. Size of deal relative to the asset size of the bank, as an indicator of the importance to the institution
  2. Unique features of architecture design
  3. The number and types of business issues it was designed to overcome
  4. Level of automation, straight-through processing or streamlining achieved as a result
  5. Level of real-time, online capability achieved as a result.
  6. Level of consistency achieved in application development capabilities or business processes
  7. Number of applications, business areas or products affected or created by this project
  8. Regulatory reporting functionalities
  9. Number and types of business processes executable into the new architecture
  10. Number, types and range of channels executable as a result of this project
  11. What were the alternatives to this approach and the features of the current project that were superior
  12. Timelines – keeping to a good implementation schedule timeline, with minimum over-runs
  13. Any additional information that should give this project an advantage over competition.
2. Best Middleware Implementation Award

Some of the considerations we look at for Middleware are the same as for core banking, but from a different perspective.

  1. Unique features of architecture design of the middleware interface
  2. The number and types of business issues it was designed to overcome
  3. Level of automation, straight-through processing or streamlining achieved as a result
  4. Level of real-time, online capability achieved as a result.
  5. Level of consistency achieved in application development capabilities or business processes
  6. Number of applications, business areas or products affected or created by this middleware
  7. Number and types of business processes executable into the new architecture
  8. Number, types and range of channels executable as a result of this project
  9. What were the alternatives to this approach and the features of the current project that were superior
  10. Timelines – keeping to a good implementation schedule timeline, with minimum over-runs
  11. Any additional information that should give this project an advantage over competition.
3. Best Data and Analytics project
  1. How enterprise-wide was this project in terms of the sources from where it is able to pull data?
  2. How inclusive was this project in being able to deal with the widest possible range of financial, risk, operational and customer intelligence information in a timely fashion?
  3. How large was the historical data migrated from existing sources and used in the current project?
  4. To what extent was this data and analytics project able to support the bank’s ability to comply with Basel II?
  5. How would you rate the data flow design of this project in the context of the robustness and reliability of the hardware infrastructure required to support it?
  6. How integrated was this project into as many of the bank's businesses?
  7. Was the solution able to integrate seamlessly to a variety of business intelligence applications?
  8. How strong would you describe the solution’s data scrubbing capability?
  9. To what extent do you think the system is able to pull out data from the backend (financials, GL) for analytics in realtime?
  10. How would you describe the quality of the solution to provide complex 3-dimensional customer anlaytics?
  11. To what extent did this project result in in tangible and quantifiable business benefits.
4. Best CRM Project
  1. To what extent was this project able to provide a single view of the customer?
  2. To what extent was this project integrated into the back-end and not just front-end transaction level touch points.
  3. To what extent was the CRM capability linked to product-level applications for this project?
  4. To what extent is this project able to support a range of data marts for transaction level, marketing, risk, profitability analytics?
  5. How would you rate the results of the marketing or sales campaigns being run using the new CRM capability?
  6. To what extent does the CRM capability provide tracking information on service fulfilment and tracking of complaints or exceptions meant for follow up action?
  7. How real-time is the data captured?
  8. To what extent was the project responsible for an improvement in cross-sell ratio per customer?
  9. To what extent was the project responsible for an improvement in profitability ratio per customer?
  10. To what extent was the project responsible for an improvement in marketing campaign turnaround time?
  11. To what extent has the CRM project resulted in increased sales, reduced risk, increased marketshare etc.?
5. Best Risk Analytics Project
  1. How enterprise-wide is this project designed to be?
  2. Is the bank able to develop its own risk scorecard for PD/EAD/LGD based on Advanced IRB requirements of Basel II as a result of this project?
  3. How robust would you assess the credit scoring, modeling and analysis capabilities in order to achieve a complete view on an enterprise's risk position? (for example, is the solution able to work out probabilities of default (PD) for various asset classes, exposure calculation and estimation and modeling of risk factor distributions from the credit scoring, modelling and analytical capabilities being implemented?)
  4. Is the system able to work out the risk weighted assets; regulatory capital, etc. according to the new Basel II?
  5. Can the system support scenario analysis, such as a view on risk management by simulating the effects of customer specified scenarios and by using all relevant techniques, such as Monte Carlo simulation, What-if? analysis, stress testing, etc.?
  6. How truly end-to-end was this project, in terms of all the relevant functionalities needed — data management, analytics, and reporting — into one architecture?
  7. What was the volume of the historical data that needed to be scrubbed and made available for this project?
  8. Does the solution enable to the bank to develop behavioural scores for each customer across product sets?
  9. How would you rate the ability of the system to pull from a variety of data pools to measure PD, LGD and EAD separately?
  10. How would you rate the number, types, timeliness and usefulness of the regulatory reports generated as a result of this project.
  11. How would you rate this solution’s ability to collect data from a range of sources.
  12. What do you think the quantifiable impact on the existing or new business in terms of lower losses, higher profitability or market share has been as a result of this new risk analytics capability?
  13. To what extent do you think the bank is able to generate an enterprise level risk dashboard as a result of this project?
6. Best Banking Product Application (can specify the application according to the business it is intended for)
  1. How important is the product application to the overall business of the bank?
  2. How integrated is the application to the back-end of the bank’s main system?
  3. How integrated is the application to the front-end of the bank’s delivery capability?
  4. How integrated is the application to other applications?
  5. How transaction intensive is this application?
  6. How realtime is this application able to be in pulling data from various sources?
  7. There was a strong level of consistency achieved in the application or business processes.
7. Best banking security system
  1. How enterprise wide is the bank’s security system?
  2. How many business critical applications does this application support?
  3. This security system supports a wide range of employee teams working in different business configurations
  4. To what extent is this system able to support realtime data?
  5. To what extent is this system able to support transaction intensive activity?
  6. To what extent is this security architecture able to provide assurance that none of the customer facing portals host malware?
8. Best ATM installation and management solution
  1. Was this solution able to demonstrable a good reduction in the average cost-per-transaction for ATMS in this network?
  2. The ATMs supported in this network have high volume (number of transactions per machine per day) configurations?
  3. This ATM network maintains a high level of uptime and availability
  4. This ATM network maintains high levels of security to support customer confidence as well as protect against unauthorised interference?
  5. This ATM network is able to be fully integrated with shared networks, regardless of platforms?
  6. This ATM network is able to support a large number of services and functionalities?
  7. This ATM network has a high level of personalisation in customer interface.
  8. This ATM network has seamless synchronisation and cross-functional support with other channels?
  9. This ATM network is able to run e-marketing solutions in conjunction with other channels
  10. This ATM network is integrated with call-center support;
  11. This network is able to support a suitable mix in terms of types of ATMs, from cash dispensers to full service terminals?
  12. The software solution supporting this network is able to access information in the core customer information file (CIF) of the bank’s data.
  13. This network is able to generate net interchange fee, simply by keeping its running cost low.
9. Best call centre project
  1. What is the size of the call center relative to others in the competition (by number of seats or volume of calls it takes)?
  2. Does the system being reviewed possess personalized voice and Call Distribution Functions?
  3. Does the system being reviewed possess Remote (Home) Agent system functionality?
  4. Does the system being reviewed possess VOIP/Web service functionality?
  5. Is the system able to integrate contact records from external sources easily?
  6. To what extent is the system able to compensate and distribute work according to the proficiency levels of the agents?
  7. Is the system able to service customers located overseas?
  8. Is the system able to facilitate Voice/Mail/E-Mail/FAX integration?
  9. To what extent are agents able to access customer transactional and personal information in the bank’s core system?
  10. What is the quality and depth of the supervisor’s reports generated by this system?
  11. How simple and quick is it to script agent’s calls and FAQs on a continuos basis?
  12. What was the improvement data in customer-contact ratios (cross-sell, profitability etc) reported by the use of this solution?
  13. How robust would you describe the quality and versatility of the queue management system?
  14. To what extent did the bank achieve strong financial performance for the business lines affected as a result of this new system?
10. Best e-banking project
  1. How would you rate the level of automation in the back-end supporting the e-banking platform?
  2. How integrated is the e-banking platform to other channels?
  3. What is the number of customer activities supported by this platform?
  4. What percentage of the bank’s total customer base are described as active on the e-banking channels?
  5. Are the functionalities provided integrated with that of other banks?
  6. This project had strong security features, including encrypted messaging, firewalls etc
  7. This project is highly scalable to support strong network or channel growth in the future
  8. This project enables the banks multiple channels to be integrated seamlessly with shared networks
  9. We are satisfied that the project has good disaster recovery features in place.
  10. We like the technical design of the e-banking platform.
11. Best multi-channel capability project
  1. To what extent were you able to detect that this project was central to the bank’s key business strategy?
  2. To what extent would you say that the distributed processing architecture of this multi-channel project was designed unique challenges (such as to overcome a large country with telecommunication challenges etc)
  3. What is the number and types of customer touchpoints that this project involves?
  4. Would you say that the architecture maintained an optimal balance between centralized processing of transactional data versus regional processing centers?
  5. The efficiency levels and cost savings of the cabling and connectivity for this project was high
  6. The storage and network configuration managed to achieve a high level of speed and response times
  7. The architecture managed to achieve good load balancing in supporting both high and low demand areas.
  8. The use of multi-protocol switching capability improved the “anytime-anywhere” capability of the network
  9. This project had strong security features, including encrypted messaging, firewalls etc
  10. This project is highly scalable to support strong network or channel growth in the future
  11. This project enables the banks multiple channels to be integrated seamlessly with shared networks
  12. We are satisfied that the project has good disaster recovery features in place.
12. Best Branch Automation
  1. This project was central to one of the bank’s important business strategy
  2. The project reduced the need for cash handling by branch staff, while increasing the ability of the branch itself to conduct cash-based transactions
  3. Cash handling achieved a high degree of automation (from teller counting to reconciling at end of day)
  4. Document handling achieved a high degree of automation (using document readers and data entry solutions) as indicated through turnaround time for Mortgage Application or similar service.
  5. The footprint of the branch was dedicated more towards customer transactions and service as a result of this project
  6. The typical benchmark time for Account Opening was reduced drastically.
  7. The security features of the branch, its cash and documentation transactions were improved as a result of this project
  8. This project resulted in reduced time for training new branch staff
  9. This project resulted in increased time available for customer servicing.
  10. The automation methodology was easily replicable across the bank’s entire branch network to ensure consistency of delivery levels.
13. Best Financial Supply Chain Project

 


14. Best HR Systems Implementation Project
  1. The number of employees and locations supported by this solution is high.
  2. There is an end-to-end recruitment Interview workflow that is tied to the organisation’s selection criteria in this system.
  3. We are impressed by the quality of the rules-based engine to handle complex functions such as Benefits administration well.
  4. The system has a clear but flexible Performance Administration System
  5. The system has an STP-driven Payroll administration system
  6. High turnaround time for 90-95% of employee administration matters
  7. The system is able to support the organisation’s human resource objectives.
  8. The system is able to support teamwork-oriented activities in the organization.
  9. There is a high degree of online, self-administered staff functions (eg. leave administration, claims admin etc)
  10. The system is able to achieve a high degree of automation for non-essential matters.
  11. The solution has a strong human resource standard operating procedures (SOP) and other functionalities
  12. There is a good staff exit management system
  13. The bank was able to achieve a strong HR system in a cost effective manner.

 

 
  Important Links
  Winners List 2011
  Photo Gallery 2011
  The Asian Banker Summit 2011
  Download the 2012 Application Form
  Academic Partner for Technology   Implementation Awards


  Home    |   About us    |   Contact us        Copyright © 2011, TAB International Pte. Ltd. All Rights Reserved.