Evaluations
Increasing need to ensure good practice has led to an increasing emphasis on monitoring and evaluation. We need to be able to measure how far projects prove to be relevant and effective. Some still take the process to be a token one while others take it very seriously. Going but not yet completely gone are the days when one set of project managers spending public money in developed countries financed another set of managers in developing countries who were asking for help and the latter then nearly always certified that it was a great success even if nothing much had emerged from the project. Future money flows are often dependent on perceived current results.
Even now undertaking an evaluation often requires Consultants to tread treacherous ground. There is usually safety in affirming that the project being evaluated was a reasonable success, no matter what. Following the easy path transforms the assignment into a pleasurable junket. There is unlikely to be a come back, no one goes back to old projects to re-evaluate them. New funds flow in time and all is forgotten.
BUT if a client or a consultant takes the task seriously and is effective, the result may be that a project is terminated mid way or there is no follow up project. It does not have to be that way since the honest conclusion may be that it was a great success, as was the case for us on the ARSP evaluation in the Philippines. We could not praise the project enough. It was also true on the ITC Generic Market Development and Promotion of Jute. The two projects were good examples of public money well spent and we said so.
When Consultants find that things have gone seriously wrong and are honest enough to say so, not every client is immediately pleased. You can be thrown out of the country by an angry Government with your client distancing themselves from you as much as possible. The implementing authority may refuse to collaborate with the consultant any further requiring his or her replacement. The client may congratulate the consultant and say the report was good, maybe too good and the consultant would be excluded from further involvement in the project.
The inevitable temptation is to avoid being Team Leader on an evaluation if discreetly possible, to be very careful of how you interpret and report findings and realise that honesty has long term rewards in professional standing but is usually quite disastrous financially in the short run. Most of all, a whistleblower is never celebrated by clients and the action must be taken only for your own personal self worth. It is very important for clients and consultants to understand the function more clearly. A good and honest and thorough evaluation is in everybody's interest and monitoring must be built into projects so that success is more likely.
We have undertaken the following Evaluations:
§ EC:
Mid Term Facility Agricultural Commodities Trade (FACT) in the Pacific. Being undertaken in January-February with visits to Fiji, Tonga, Vanuatu, Solomon Islands and papua New Guinea. FACT is working with 15 selected enterprises to assist them to develop their exports. A larger project, IACT, is intended to follow up the current one and part of the evaluation is to make recommendations as to what it should include.
The Evaluation was largely positive. Under the able leadership of Dr Lex Thomson, a committed and enthusiastic team is undertaking their work with vigour and vision.
Vinay and Lex stand beside the Rabaul crater in PNG.
Mid Term Kerafed (Team Leader) for Kerala coconut industry to increase incomes of farmers and processors through higher field productivity, off farm employment and agri-businesses;
The Kerafed project was a grant assisted project in Kerela, India. The European Commission pledged 45 million euros as a grant. The objective was poverty alleviation among coconut farmers in Kerela. Three oil mills processing 20 million coconuts each per annum were planned together with assistance to copra drying and coconut agronomy.The project fell under the ageis of the National Cooperative Development Corporation (NCDC).
The projective objectives included 3 oil mills, a solvent extraction plant and a refinery. The project ran late due to a variety of reasons. It also ran into trouble in the form of problems concerning project implementation.
A Team consisting of 3 International Consultants and one local Consultant were recruited in 1992 for the Mid Term evaluation. Vinay Chand was Economist/Team Leader, Francoise Rognon the Cooconut Agronomist and Jean Fanguin, Oilseeds Engineer.
The Mission identified the serious problems that the project had run into and called for changes. In the end Kerafed decided to carry on with funding from the Indian Government and is running today although it was and probably still has to be subsidised. The pursuit of value addition has to ensure that there is a net increase in value. Subsidies too need to be carefully monitored and, in India, subsidies for copra and protection for coconut oil has discouraged much higher end value use as coconut milk.
The worst aspect of the project was that in having to terminate it, the EC also aborted another project in the pipeline that would have greatly assisted the coconut farmers. The result was not a good indicator for India.
---------------------------------------------------------------------------------------------------------------------
Pre Completion Agrarian Reform Support Programme (ARSP)
ARSP Philippines was a Grant assisted EC project with a contribution of 20 million Euros. It was sited in 5 southern Provinces of the country: Camarines del Sur; Negros Oriental; and Agusan del Norte and Agusan del Sur in Northern Mindanao.
Farmers being settled on allocated land were assisted in gaining title to the land, were organised into farmer groups that were given training, basic infrastructure was provided for the groups and some were provided primary processing machinery. Nurseries were established to supply planting material. There was Technical assistance and a revolving credit was channelled through local banks in which the farmer groups acquired equity.
The 5 year programme was run by a Team Leader from Finland and a co-Director from the Philippines.
The Mission was composed of Hans Meliczek, Team Leader, Mr. Vinay Chand, Agricultural Development Specialist, Mr. Jean Noel Perrin – Economist / Credit Specialist and Mrs. Virginia Cardenas, Sociologist / Gender Specialist.
Although the evaluation was intended to be a pre-completition Mission, the project had actually been completed in 2003 when the Mission took place. A latge number of beneficiaries and others involved were interviewed and the unanimous feeling was that the project had been a great success. Farmer incomes, a key criteria, had risen very substantially.
One reason for evaluations is to look for good projects that can be replicated. This one certainly should be. Unfortunately, replication also depends on country programmes and donor strategy and the ARSP was not replicated in the Philippines. Other donors had similar projects but they were not grant based ones and that element was very important in the success of the project.
-------------------------------------------------------------------------------------------------------
NORAD:
Post Completion Evaluation (Bangladesh Jute Development) (Team Leader) which had linked three jute mills with NGOS and poor women’s rural groups.
To further diversification, NORAD started helping Bangladesh produce jute fabrics and making them softer, less hairy and shedding less fibres. The idea was to help diversify jute exports while also providing poor rural women with yarn, fabrics and design assistance to produce soft furnishings that would appeal in export markets.
The programme evolved as needed and not according to any preconceived plan, with the Private Sector Development Team at NORAD working directly with interested private sector participants in Bangladesh with support of the Bangladesh Jute Spinners Association. The approach was new for NORAD and a learning process ensued. With changing personnel and dual objectives of helping generate income for rural women and enabling substantial diversification, there were bound to be problems but at the end of the day both objectives were successfully furthered.
Vinay was sole evaluator in 2003 and the report acknowledged that the project had been successful but needed further assistance to become commercially viable. NORAD arranged for SEDF to look into this. Project members had succeeded in developing a soft material and women in rural areas had transformed the material into cushion covers and other similar household items. These were being sold under exclusive brand names. In order for the operation to increase scale substantially it would be necessary to lower production costs and gain access to larger volume outlets.
---------------------------------------------------------------------------------------------------------------------------------------------
SEDF/IFC:
Mission was undertaken to evaluate global trends in jute consumption and production in Bangladesh to make recommendations as to possible follow up action through IFC ;
A follow up on the NORAD project above, Vinay Chand, assisted by Gordon McKie was asked to evaluate the results of aid projects to the Bangladesh Jute Industry and further interventions required. What resulted was an assessment of how far the industry had developed and possible directions for growth.
Although there are excellent opportunities for diversification and sometimes for traditional products, the domestic market in India has proven to be a magnet for use of jute in India as well as some of the jute from Bangladesh.
As a result, use of jute has been allowed to decrease in international markets without resistance and there is a lack of new initiatives.
Although the report recommended actions that would achieve commercial viability and thus sustainability, there was nothing much the SEDF could do as NORAD was no longer involved.
-----------------------------------------------------------------------------------------------------------------------------
ITC
Post Completion (20 years generic Jute Market Development and Promotion) (Team Leader) to prevent deterioration in earnings of 8 million poor farmers in Bangladesh and India.
For 10 years ITC (UNCTAD/WTO) had undertaken projects to develop markets for and promote jute and jute products. They did so during a period in which synthetic fibres were challenging traditional jute markets. The jute industry was in peril and producers, particularly India and Bangladesh meeded help.
In 1999 Vinay Chand was asked to carry out an independent evaluation of the 20 years of intervention. Despite the relatively modest amount of money spent, the evaluation found that ITC had probably helped slow down the decline of the industry, time during which Indian consumption was to grow to offset the global decline.
The ITC interventions are very interesting as examples of intervention to protect demand for a commodity and its diversification into higher value products. Generic promotion of commodities has not been aggressively pursued since these interventions. In fact, most institutions today explicitly avoid generic programmes let alone market promotion.
---------------------------------------------------------------------------------------------------------------------------------
World Bank:
Evaluation Feasibility studies Vanuatu and PNG for IFC/World Bank (Sole Evaluator) to increase farmer incomes through value addition.
Vinay Chand went to Papua New Guinea to evaluate feasibility studies submitted to IFC for investment. The main proposal was from Rabaul but there was also one from Madang for an Integrated Coconut Processing system that would have produced rubberised coir mattresses, coconut milk, cream and virgin oil and activated carbon.
Papua New Guinea has amongst the lowest prices for coconuts when compared to other major producers. This makes the possibility of value added processing very attractive. However, the cost advantage has not been exploited and the evaluations represented an opportunity to promote processing to interested parties in the private sector.
He combined the Mission to evaluate a proposal from Spiritos Santos in Vanuatu for activated carbon from coconut shells by the leading local trader who had access to sufficient volumes of coconut shell charcoal to produce activated carbon (shells from 60 million coconuts per annum).
Both studies had been sponsored by individuals on part of SMEs and thus depended on continuing impetus by the same indviduals. Unfortunately, a volcano, a heart attack and an early death put paid to both the projects. The losers were the two countries. For IFC it was part of a learning curve.
---------------------------------------------------------------------------------------------------------------------------------------------