Imagine we are a translation provider for an organization like the World Bank , we heard about this new technology of neural machine translation and we would like to try out how well this works for the materials we have to translate. We do have access to some translated PDF files in English and German from years past, but unfortunately no access to a translation memory. To evaluate machine translation objectively with automated metrics like BLEU we need about 1000 to 2000 aligned, high-quality translated sentences that are representative of the material we intent to translate. In this blog post we create such evaluation data from the PDFs by extracting the text and manually aligning the sentences. In the next blog post we use this evaluation data to evaluate the translation quality of different MT systems using automated metrics. Downloading World Bank Open Knowledge Repository PDF Files Most of the World Bank Open Knowledge Repository is generously licensed under Creative C...