# Table of Contents - [FME + Any AI Training Course | Avineon Tensing UK - FME + Any AI Training Course](#fme-any-ai-training-course-avineon-tensing-uk-fme-any-ai-training-course) - [Course Resources | Avineon Tensing UK - FME + Any AI Training Course](#course-resources-avineon-tensing-uk-fme-any-ai-training-course) - [About this Document | Avineon Tensing UK - FME + Any AI Training Course](#about-this-document-avineon-tensing-uk-fme-any-ai-training-course) - [Course Overview | Avineon Tensing UK - FME + Any AI Training Course](#course-overview-avineon-tensing-uk-fme-any-ai-training-course) - [Avineon Tensing | Avineon Tensing UK - FME + Any AI Training Course](#avineon-tensing-avineon-tensing-uk-fme-any-ai-training-course) - [Chapter 1: AI Foundations & FME Integration | Avineon Tensing UK - FME + Any AI Training Course](#chapter-1-ai-foundations-fme-integration-avineon-tensing-uk-fme-any-ai-training-course) - [Prompting Techniques | Avineon Tensing UK - FME + Any AI Training Course](#prompting-techniques-avineon-tensing-uk-fme-any-ai-training-course) - [Tensing FME UK Tour 2025 | Avineon Tensing UK - FME + Any AI Training Course](#tensing-fme-uk-tour-2025-avineon-tensing-uk-fme-any-ai-training-course) - [Google Gemini Connectivity | Avineon Tensing UK - FME + Any AI Training Course](#google-gemini-connectivity-avineon-tensing-uk-fme-any-ai-training-course) - [OpenAI, Gemini - API Setup | Avineon Tensing UK - FME + Any AI Training Course](#openai-gemini-api-setup-avineon-tensing-uk-fme-any-ai-training-course) - [Course Etiquette | Avineon Tensing UK - FME + Any AI Training Course](#course-etiquette-avineon-tensing-uk-fme-any-ai-training-course) - [Chapter 2: Building AI-Powered Workflows in FME | Avineon Tensing UK - FME + Any AI Training Course](#chapter-2-building-ai-powered-workflows-in-fme-avineon-tensing-uk-fme-any-ai-training-course) - [Google Gemini | Avineon Tensing UK - FME + Any AI Training Course](#google-gemini-avineon-tensing-uk-fme-any-ai-training-course) - [OpenAI | Avineon Tensing UK - FME + Any AI Training Course](#openai-avineon-tensing-uk-fme-any-ai-training-course) - [Exercise 1.1 - The OpenAIConnector | Avineon Tensing UK - FME + Any AI Training Course](#exercise-1-1-the-openaiconnector-avineon-tensing-uk-fme-any-ai-training-course) - [Create a Roboflow Account | Avineon Tensing UK - FME + Any AI Training Course](#create-a-roboflow-account-avineon-tensing-uk-fme-any-ai-training-course) - [The Power of Custom Transfomers | Avineon Tensing UK - FME + Any AI Training Course](#the-power-of-custom-transfomers-avineon-tensing-uk-fme-any-ai-training-course) - [Upload Your Data | Avineon Tensing UK - FME + Any AI Training Course](#upload-your-data-avineon-tensing-uk-fme-any-ai-training-course) - [Create Your First Project | Avineon Tensing UK - FME + Any AI Training Course](#create-your-first-project-avineon-tensing-uk-fme-any-ai-training-course) - [Exercise 3.1 - Local Hello World | Avineon Tensing UK - FME + Any AI Training Course](#exercise-3-1-local-hello-world-avineon-tensing-uk-fme-any-ai-training-course) - [Chapter 3: Local AI | Avineon Tensing UK - FME + Any AI Training Course](#chapter-3-local-ai-avineon-tensing-uk-fme-any-ai-training-course) - [Exercise 2.2 - Parsing a JSON Response | Avineon Tensing UK - FME + Any AI Training Course](#exercise-2-2-parsing-a-json-response-avineon-tensing-uk-fme-any-ai-training-course) - [Exercise 3.4 - Sentiment Analysis | Avineon Tensing UK - FME + Any AI Training Course](#exercise-3-4-sentiment-analysis-avineon-tensing-uk-fme-any-ai-training-course) - [Exercise 2.1 - Mastering the HTTPCaller | Avineon Tensing UK - FME + Any AI Training Course](#exercise-2-1-mastering-the-httpcaller-avineon-tensing-uk-fme-any-ai-training-course) - [Exercise 3.2 - GDPR Checker | Avineon Tensing UK - FME + Any AI Training Course](#exercise-3-2-gdpr-checker-avineon-tensing-uk-fme-any-ai-training-course) - [OpenAI Connectivity | Avineon Tensing UK - FME + Any AI Training Course](#openai-connectivity-avineon-tensing-uk-fme-any-ai-training-course) - [Exercise 3.3 - Local AI Translator | Avineon Tensing UK - FME + Any AI Training Course](#exercise-3-3-local-ai-translator-avineon-tensing-uk-fme-any-ai-training-course) --- # FME + Any AI Training Course | Avineon Tensing UK - FME + Any AI Training Course [NextAvineon Tensing](/fme-+-any-ai-training-course/fme-+-any-ai-training-course/avineon-tensing) Last updated 15 days ago This is the manual for the FME + Any AI Training Experience for Safe Software’s FME Platform. Here we'll discuss the practicalities of adding value to your data integration orchestrations by sprinkling a little Artificial Intelligence (AI) into your FME workspace. This training course builds on Avineon Tensing's FME Form Introductory Training Experience to cover functionality that's important to all FME users wishing to take their data integration and AI skills to the next level. For this course we do recommend that you have at least a basic understanding of FME Form to get started, some AI integrations are more complex than others, but we aim to focus on no-code implementations where possible. This course is an introduction to the practical use of AI within an Extract, Transform and Load (ETL) or Data Integration project and we hope to inspire you to use AI components within your data workflows afterwards! [](#course-structure) Course Structure ------------------------------------------- The full course is made up of seven sections. These sections are: * Chapter 1: AI Foundations & FME Integration * Chapter 2: Building AI-Powered Workflows in FME * Chapter 3: Local AI * Chapter 4: AI-Driven Image Processing with Roboflow * Chapter 5: FME Orchestration - Generative AI and Computer Vision * Chapter 6: Handling Dynamic Schemas in AI Outputs * Chapter 7: Unlocking Access to your Data [](#current-status) Current Status --------------------------------------- The current status of this manual is: **COMPLETE**: this manual **can** be used for training. It is valid for FME Form version **FME 2025.0.1.0 - Build 25220** The status of each chapter is: * Chapter 1: Complete * Chapter 2: Complete * Chapter 3: Complete * Chapter 4: Complete * Chapter 5: Complete * Chapter 6: Complete * Chapter 7: Complete * Training data: Complete _**NB:**_ _Even for completed content Avineon Tensing (Avineon Europe Ltd) assume no responsibility for any errors in this document or their consequences, and reserves the right to make improvements and changes to this document without notice. See the full licensing agreement for further details_ _._ [_https://www.safe.com/legal/_](https://www.safe.com/legal/) FME Form, part of the FME Platform from Safe Software ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fcontent.gitbook.com%2Fcontent%2F3gaZkHJOAvL5gXLEsKl6%2Fblobs%2FZVnJFVMXWlmebUvjMvlD%2Fimage.png&width=768&dpr=4&quality=100&sign=954bbb4b&sv=2) --- # Course Resources | Avineon Tensing UK - FME + Any AI Training Course A number of sample datasets and Workspaces will be used in this Workshop. [](#on-your-training-computer-or-strigo-lab) On Your Training Computer or Strigo Lab ----------------------------------------------------------------------------------------- The data used in this course is based on open data from a number of different global sources or completely fictitious data that we have fabricated for the purposes of demonstration. Whether it's a local computer or a virtual computer hosted in the cloud, you'll find resources for the examples and exercises referenced in the workshop material at the following locations and your trainer will help you understand how to locate it: Location Resource C:\\TrainingData\\Data Source datasets C:\\TrainingData\\Workspaces Workspaces used in the exercises C:\\TrainingData\\Output The location to write exercise output, so you can access it after the Workshop You should also find FME Form pre-installed and licensed plus you'll be issued a copy of this manual. Please alert your trainer if any item is missing from your setup. [PreviousCourse Overview](/fme-+-any-ai-training-course/fme-+-any-ai-training-course/course-overview) [NextCourse Etiquette](/fme-+-any-ai-training-course/fme-+-any-ai-training-course/course-etiquette) Last updated 16 days ago --- # About this Document | Avineon Tensing UK - FME + Any AI Training Course The material has been developed exclusively by Avineon Tensing and the associated training data and examples will be managed by Avineon Tensing (Avineon Europe Ltd). [](#licensing-and-warranty) Licensing and Warranty ------------------------------------------------------- Permission is hereby granted to the individual to whom this content has been issued to use the manual and associated training data for the purposes of learning and personal development after the formal training course has been delivered. Permission is not granted for the modification or redistribution of this material to persons beyond those who attended the Avineon Tensing FME training course. Safe Software Inc. and Avineon Tensing (Avineon Europe Ltd) makes no warranty either expressed or implied, including, but not limited to, any implied warranties of merchantability, non-infringement, or fitness for a particular purpose regarding these Tutorials, and makes such Tutorials available solely on an “as-is” basis. In no event shall Safe Software Inc. or Avineon Tensing (Avineon Europe Ltd) be liable to anyone for direct, indirect, special, collateral, incidental, or consequential damages in connection with or arising out of the use, modification or distribution of these Tutorials. This manual describes the functionality and use of the software at the time of publication. The software described herein, and the descriptions themselves, are subject to change without notice. [](#copyright) Copyright ----------------------------- © 2025 - Avineon Tensing (Avineon Europe Ltd). All rights are reserved. [](#trademarks) Trademarks ------------------------------- FME® is a registered trademark of Safe Software Inc (Vancouver, Canada). All brand or product names are trademarks or registered trademarks of their respective companies or organisations. [PreviousTensing FME UK Tour 2025](/fme-+-any-ai-training-course/fme-+-any-ai-training-course/tensing-fme-uk-tour-2025) [NextCourse Overview](/fme-+-any-ai-training-course/fme-+-any-ai-training-course/course-overview) Last updated 16 days ago --- # Course Overview | Avineon Tensing UK - FME + Any AI Training Course **Transform Chaos into Clarity: Use FME and AI to Extract, Convert, and Classify Unstructured Data** Structured data extraction is a key capability of Large Language Models (LLMs). Providers like OpenAI, Anthropic, and Gemini now offer APIs that transform unstructured content into clean JSON outputs. But LLMs alone aren’t enough. Without robust tools to automate, validate, and scale the process, you’re left with fragile scripts and manual clean-up. That’s where FME comes in - seamlessly integrating generative AI into real-world data workflows. [](#prerequisites) Prerequisites ------------------------------------- This course is delivered at an introductory through intermediate level and whilst prior experience of FME Form is useful, it's not mandatory, therefore there are no prerequisites for attendance of this course. [](#about-the-manual) About the Manual ------------------------------------------- The Training manual not only forms the basis of the training – in-person or online – but is also useful reference material for future work you may undertake with FME. It is updated for each major release of FME by the specialist FME Certified training team at Avineon Tensing. All screenshots in these materials were taken using FME on a Windows Operating System. The fonts used (especially in screenshots of the log window) may be resized or otherwise changed for improved legibility. **Some screenshots included in the manual may be subtly different to what you see in your locally installed FME software, this may be due to a difference in FME version and in this case, you are advised to discuss the differences with your trainer.** [PreviousAbout this Document](/fme-+-any-ai-training-course/fme-+-any-ai-training-course/about-this-document) [NextCourse Resources](/fme-+-any-ai-training-course/fme-+-any-ai-training-course/course-resources) Last updated 15 days ago --- # Avineon Tensing | Avineon Tensing UK - FME + Any AI Training Course [PreviousFME + Any AI Training Course](/fme-+-any-ai-training-course) [NextTensing FME UK Tour 2025](/fme-+-any-ai-training-course/fme-+-any-ai-training-course/tensing-fme-uk-tour-2025) Last updated 16 days ago At Avineon Tensing, we future-proof your geospatial data management with continuous support for evolving formats, services, and standards. Our GIS, FME, and data engineering specialists deliver scalable, innovative solutions. We transform geospatial data into a strategic advantage that drives smarter decisions, actionable insights, and measurable outcomes. [](#expertise-you-can-rely-on) Expertise You Can Rely On ------------------------------------------------------------- With over 30 years of combined expertise, we bring together deep technical knowledge in geospatial systems and hands-on experience with data-driven solutions. Whether you need to integrate complex datasets, develop pipelines for cloud and hybrid environments, or adopt cloud-native formats for optimal data storage and delivery, we provide the tools and expertise to ensure your data management is ready for the future. Operating from five offices across Europe, in Lier (BE), London (UK), Montpellier and Paris (FR) and Utrecht (NL), our team works closely with you to align solutions with your goals. From initial concept to long-term maintenance, we guide you through all stages of geospatial data management, helping your organisation adapt and thrive in a rapidly evolving field. [](#specialists-in-fme-and-no-code-solutions) Specialists in FME and No-Code Solutions
 -------------------------------------------------------------------------------------------- As a Safe Software Value-Added Reseller (VAR) Partner, we provide comprehensive support for FME Platform solutions. This includes tailored deployment solutions that align with your data and infrastructure, as well as flexible, interactive training delivered by certified experts. We guide you in implementing Extraction, Transformation, and Load (ETL) processes using FME Form and Flow, enabling no-code workflows for geospatial data integration, transformation, and delivery. Our expertise also includes leveraging advanced approaches such as no-code/low-code solutions to simplify data integration and visualisation. Additionally, we apply Generative AI (GenAI) to optimise data cleanup and augmentation workflows. These tools ensure your data is accurate, actionable, and ready to meet your organisation’s unique needs. By combining these tools with our knowledge of the geospatial landscape, we help you streamline workflows and maximise the value of your data assets. [](#driving-innovation-and-empowering-growth) Driving Innovation and Empowering Growth ------------------------------------------------------------------------------------------- At Avineon Tensing, we specialise in building solutions that grow with your organisation. Our end-to-end support ensures your data management processes are agile and scalable, adapting to new formats, standards, and demands. Whether you’re preparing for future challenges or seeking to optimise your current operations, we empower your team to stay ahead of competition. With a commitment to innovation and collaboration, we also provide unmatched training and support to strengthen your team’s capabilities. By aligning innovative tools with personalised guidance, we ensure that your organisation not only manages geospatial data efficiently but also realises its full potential with measurable outcomes. [](#your-trusted-partner-in-geospatial-excellence) Your Trusted Partner in Geospatial Excellence ----------------------------------------------------------------------------------------------------- Partner with Avineon Tensing to explore new opportunities, exceed user expectations, and thrive in an evolving geospatial landscape. Whether you need to modernise your workflows, enhance your team’s expertise, or adopt new technologies, we’re here to guide you every step of the way. Together, we unlock the full potential of your geospatial data. Now and in the future. Contact us at +44 (0)20 8058 8106 or uk@avineon-tensing.com. [](#whats-in-a-name) What's in a name? ------------------------------------------- Consider us as your trusted guide and advisor. We want to support you to be more efficient and, if you want to be, more self-sufficient at managing your own data. It's why we hold such great stock in training and personal development. [](#what-do-we-do) What do we do? -------------------------------------- ### [](#fme-platform-licences) FME Platform Licences [](#professional-services) Professional Services ----------------------------------------------------- If you need help deploying FME, or using FME... or you'd like us to build one of your FME processes, we can help. ### [](#training-and-coaching) Training and Coaching Our FME training is designed to help you grow, both personally and professionally and to support that, we’ve developed a training programme to support your data integration journey with the FME Platform. Whether it’s authoring format translation processes with FME Form, or configuring an automation, triggered by an event in your data ecosystem using FME Flow or FME Flow Hosted, or perhaps something more bespoke… we’ve got it covered. FME training helps users stay up-to-date with the latest features and techniques. As the software continues to evolve, users who engage in ongoing formal training are better equipped to leverage new capabilities and features, ensuring they remain competitive in the ever-changing field of data management. Training can be fun too… did we mention that? We cover the full range of FME training, from beginner to advanced and with standard and bespoke courses, delivered online or face to face. Just select the approach that best fits your needs and get in touch with us to discuss your requirements. We sell FME licences. It's the first step to accessing FME. Let our FME Certified Business Professionals; and help you get access to the FME Platform in the most cost effective way. [Oliver](https://www.tensing.com/en/over-ons/team/oliver-morris) [David](https://www.tensing.com/en/over-ons/team/dave-eagle) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fcontent.gitbook.com%2Fcontent%2F3gaZkHJOAvL5gXLEsKl6%2Fblobs%2FqdggLMG3246nQDLxTJEj%2FIMG_20231025_173619_v2.jpg&width=768&dpr=4&quality=100&sign=ab2a9372&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fcontent.gitbook.com%2Fcontent%2F3gaZkHJOAvL5gXLEsKl6%2Fblobs%2FSKaRIVkzvv3c41kA9f3i%2Fimage.png&width=768&dpr=4&quality=100&sign=ab358758&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fcontent.gitbook.com%2Fcontent%2F3gaZkHJOAvL5gXLEsKl6%2Fblobs%2Fsr05OOBowmDRsGfzQsIV%2FAvineon-Tensing%2520logo_landscape_RGB.jpg&width=768&dpr=4&quality=100&sign=98f586dd&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fcontent.gitbook.com%2Fcontent%2F3gaZkHJOAvL5gXLEsKl6%2Fblobs%2FHVoKJmgL6PGuAPMTwBLT%2FVAR_logo_artwork.rgb%2520%282%29.png&width=768&dpr=4&quality=100&sign=6338d2f2&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fcontent.gitbook.com%2Fcontent%2F3gaZkHJOAvL5gXLEsKl6%2Fblobs%2FS1DVEFyoUKQGKETfXn5u%2Fimage.png&width=768&dpr=4&quality=100&sign=50671262&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fcontent.gitbook.com%2Fcontent%2F3gaZkHJOAvL5gXLEsKl6%2Fblobs%2FJYFfowJDhUeyfL0rq68P%2Fimage.png&width=768&dpr=4&quality=100&sign=af94e582&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fcontent.gitbook.com%2Fcontent%2F3gaZkHJOAvL5gXLEsKl6%2Fblobs%2FYBrtsXyci0LJMDTxP42o%2Fimage.png&width=768&dpr=4&quality=100&sign=a78d175e&sv=2) --- # Chapter 1: AI Foundations & FME Integration | Avineon Tensing UK - FME + Any AI Training Course [PreviousCourse Etiquette](/fme-+-any-ai-training-course/fme-+-any-ai-training-course/course-etiquette) [NextPrompting Techniques](/fme-+-any-ai-training-course/chapter-1-ai-foundations-and-fme-integration/prompting-techniques) Last updated 15 days ago [](#introduction) Introduction ----------------------------------- The field of generative artificial intelligence encompasses a multitude of providers, each offering a diverse array of models. These providers often cultivate distinct specialisations, leading to variations in their core competencies. Furthermore, the individual models within each provider's portfolio are typically designed and optimised for specific applications. Notable examples include OpenAI, whose GPT series of models has demonstrated significant capabilities in text and code generation. Google offers models such as Gemini, exhibiting proficiency across a range of modalities. Additionally, Stability AI (a UK-based artificial intelligence company) has made substantial contributions in the domain of image synthesis with it's best known text-to-image model Stable Diffusion. Consequently, the current generative AI ecosystem presents a broad spectrum of providers and models, each possessing unique strengths tailored to particular tasks. ### [](#getting-the-best-from-ai-with-fme) Getting the best from AI with FME The FME Platform is purpose built to facilitate data connections, those connections can be to files or databases, cloud data stores or APIs. As AI services invariably offer an API endpoint of some type, with the FME Platform you don't have to limit yourself to just one AI model or provider. Users can leverage the distinct strengths of individual AI models as part of a cohesive data processing pipeline. For example, in a _single process_ you could employ a natural language processing model from **OpenAI** for text extraction, subsequently pass the results to a Local AI Model hosted on **Ollama**, in order to work with sensitive data, then ping some images that are associated with the text off to **Google** Gemini Vision for some analysis and then use FME to load a Digital Asset Management system with both the images and the text. Through your orchestration process you have managed to enrich the data with the right AI service for each task! The FME Platform's inherent flexibility allows for the construction of sophisticated, multi-stage data workflows, incorporating AI at a point when it can add real value to your process and hopefully optimising the overall data integration process by harnessing the specific capabilities of the right AI technologies at the right time! ### [](#models) Models An AI 'model', at its heart, is essentially a computer program that's been trained on a massive amount of data. Think of it like teaching a very clever dog about lots and lots of examples of something, perhaps showing it lots of pictures of cats! After seeing enough examples, the dog (or the model) learns to recognise a cat when it sees one, and it knows when it sees a racoon, though it may not care to differentiate it's response! In the AI world, instead of pictures, it could be text, images, sounds, or anything really. The 'training' process allows the model to identify patterns and relationships in the data, which it then uses to make predictions, generate new content, or understand information. So, when you hear about different AI models, they've just been trained on different types of data and for different, often very specific tasks. A real world example of this kind of activity sometimes makes it into the news: ### [](#example-models) Example Models Models can be very different in capabilities and also in respect to cost. At Avineon Tensing we recommend that you do some of your own research before you adopt a model, do your own due diligence on things such as: * How much it's going to cost you to use; * What data it was trained on; * What the vendor will do with the data you supply it (even if you are just querying a model you can give away sensitive data); * What the legalities are for using it, for example, are you allowed to use it for commercial projects. As an example here are some differences in model capabilities and best use cases: **Aspect** **Provider** OpenAI (independent AI research org) Microsoft (via Azure, partnered with OpenAI) Google **Key Models** GPT-4o, o1 series, DALL-E, Whisper GPT-4o, o1 series, DALL-E, Whisper Gemini 2.0 and 2.5 (Pro, Flash) **Accessibility** ChatGPT, API Azure cloud platform, API Google Cloud, Vertex AI, AI Studio, API **Strengths** Advanced NLP, reasoning, multimodal (text, image) Enterprise-grade security, Azure integration Multimodal (text, image, video), large context window **Context Window** Up to 128k tokens (GPT-4o) Up to 128k tokens (GPT-4o) Up to 2M tokens (Gemini 2.5 Pro) **Pricing** Token-based, $20/month ChatGPT Plus Pay-as-you-go or Provisioned Throughput Units Competitive, e.g., Gemini Pro ~$0.00025/1k tokens **Use Cases** Chatbots, content creation, research Enterprise apps, secure workflows, analytics Industry-specific solutions, large docs We’ll explore how to prepare effective prompts for AI services and how to configure some of the AI transformers in FME to help us generate the output we require. ### [](#learning-outcomes) Learning Outcomes: In this chapter we'll learn: * How to effectively write a prompt, providing enough context to get a good answer. * How to access a couple of the main players in the AI space; OpenAI and Google's Gemini service. * Understand how to interact with different AI models in FME. **OpenAI** **(Google)** ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252FYswb5PrnBDmaju69Dhc8%252Fimage.png%3Falt%3Dmedia%26token%3D400bfada-5077-4e59-8fba-ee1d6f9abff9&width=768&dpr=4&quality=100&sign=b4897e1f&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252FBNYVYljHW1yzDKDw8RM4%252Fcat-racoon.png%3Falt%3Dmedia%26token%3D5173460e-3b89-4a96-9c41-09685b754831&width=768&dpr=4&quality=100&sign=7f4d66c9&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252FqR2ALoU9NfofbPUQeX6b%252Fimage.png%3Falt%3Dmedia%26token%3D4bbf1b77-84fb-481c-b7e5-ea005fda80ea&width=768&dpr=4&quality=100&sign=8f21d238&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252FRF4iNLZlFu1F2x9TLLWO%252Fimage.png%3Falt%3Dmedia%26token%3D5dbd0068-df84-4ce4-a672-18f69a6f1da8&width=300&dpr=4&quality=100&sign=fa7ca739&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252FB08GLY5MLVzKxv1eSk2r%252Fimage.png%3Falt%3Dmedia%26token%3D0a01b392-34ad-451e-9c83-3b7451b4679c&width=300&dpr=4&quality=100&sign=79170a1&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252FhHisfrAHwBpokiaQn6gp%252Fimage.png%3Falt%3Dmedia%26token%3D54b4e267-dcbd-478a-8c1e-8832b9b2b355&width=300&dpr=4&quality=100&sign=96521387&sv=2) --- # Prompting Techniques | Avineon Tensing UK - FME + Any AI Training Course Welcome to the wonderful world of prompting, or "**prompt engineering**" to give you it's more grandiose title! Just as you wouldn't start building a workspace without knowing at least a little about your source data, effective prompting begins with clear instructions to guide your AI assistant toward the output you need. ### [](#understanding-prompts) Understanding Prompts Think of a prompt as the blueprint for your AI interaction. Like a well-designed FME workspace, a good prompt combines structure and flexibility. Prompts generally include text, but increasingly images and sometimes even audio or video components can be issued to your AI of choice. In our data transformation world, mastering prompting can help you accomplish tasks ranging from quick script generation to complex data analytics - saving you valuable time, whilst ensuring the AI understands: * exactly what you're after, and; * returns a response that you can practically do something with! ### [](#prompting-approaches) Prompting Approaches #### [](#direct-prompting) Direct Prompting Straightforward and to the point. > "Write a Python script to read a CSV file." Sounds simple, right? However, direct prompts often need more support and context for complicated tasks. #### [](#contextual-prompting) Contextual Prompting When you provide background information and specific details, you're helping the AI understand the complete picture. This is like adding annotations to your workspace - it creates clarity and helps to prevent misinterpretations. #### [](#iterative-prompting) Iterative Prompting Few workspaces run perfectly on the first try, and the same goes for prompting an AI. Start broad, test the output, then refine. Each interaction builds on the previous one, gradually transforming your rough request into a finely tuned solution - just like how you might debug a workspace step by step. You might for example have seen the trend recently on social media to transform a likeness of yourself into an action figure, like our colleague in Canada, Kenneth: You'll notice that many of the people who tried it out, explain in their post that they often needed multiple attempts to get an acceptable result. The first few tries might have resulted in the figure not wholly inside the blister pack, or the text on the box being in the wrong place. An iterative approach meant that for some at least, a more context rich prompt could be developed, along the lines of: > "Create an image of an action figure in packaging labeled '**\[Your Label\]**'. Use the attached photo as a reference for the face. **\[He/She/They\]** is **\[Height\]** tall and dressed in **\[Outfit Description\]**, and holding **\[Item, e.g., Coffee Mug\]**. The cardboard section should be **\[Color\]**. Include an 'Accessories' section with items like **\[List Accessories\]**. Make the design visually appealing and reflective of a **\[Describe Industry\]**. Do not include the character image on packaging." In fact, a thread has started on this topic in the FME Community too, as a result of their Question of the Week series! ### [](#best-practices) Best Practices Stepping away from a trivial scenario though, in a data integration context there are techniques that will get you to your goal more quickly when writing a solid prompt. For example, you should: * Tell the AI what data formats you're working with and what output you need. * Provide context about your overall workflow goal. The more context you provide, the better the results. * Share examples when possible - if you're asking for a specific output or a specific output structure, showing "before and after" samples helps tremendously. * For complex questions, ask for step-by-step explanations to see what's happening at each stage. There are also some strategies that will help you, for example, consider: * Testing your prompts in a chat environment before relying on them for production work. * Utilise system prompts to set the scene and give additional context to your request. * Include visual references when needed - screenshots or sample data can clarify your requirements better than words alone. **Pro tip:** Consider creating a prompt library to share successful prompts with your colleagues. It'll save everyone time and ensure consistent results! * * * ### [](#google-prompt-engineering-guide) Google - Prompt Engineering Guide Don't just take our word for it though, Google just published a 68-page ultimate prompt engineering guide (Focused on API users). So, whether you're technical or non-technical, this might be one of the most useful prompt engineering resources out there right now. It goes deep on structure, formatting, configuration settings and real examples. Here’s what it covers: * How to get predictable, reliable output using temperature, top-p, and top-k * Prompting techniques for APIs, including system prompts, chain-of-thought, and ReAct (i.e. reason and act) * How to write prompts that return structured outputs like JSON or specific formats Grab the complete guide PDF here: Prompt Engineering Whitepaper (Google, 2025) [PreviousChapter 1: AI Foundations & FME Integration](/fme-+-any-ai-training-course/chapter-1-ai-foundations-and-fme-integration) [NextOpenAI, Gemini - API Setup](/fme-+-any-ai-training-course/chapter-1-ai-foundations-and-fme-integration/openai-gemini-api-setup) Last updated 15 days ago [Question of the Week: Create Your FME Action Figure | Community](https://community.safe.com/question-of-the-week-60/question-of-the-week-create-your-fme-action-figure-38041) [Prompt EngineeringKaggle](https://www.kaggle.com/whitepaper-prompt-engineering) ![Logo](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fwww.kaggle.com%2Fstatic%2Fimages%2Ffavicon.ico&width=20&dpr=4&quality=100&sign=d9ced9e&sv=2) ![Logo](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fuploads-us-west-2.insided.com%2Fsafesoftware-en%2Fattachment%2F3fe6c75e-77af-44c8-a0d2-cd914e34345e_thumb.png&width=20&dpr=4&quality=100&sign=28a9b5e7&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252FS2rjc9wBBN0Pzi0AWndG%252Fimage.png%3Falt%3Dmedia%26token%3D57b5586a-c1e7-4c2e-a76c-1b204f436a0e&width=768&dpr=4&quality=100&sign=b7a5ada1&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252F7MVrXOkslgpOSEA6iNNG%252Fimage.png%3Falt%3Dmedia%26token%3Dcbcef0e3-3b90-438a-a6d4-90aae86f709b&width=768&dpr=4&quality=100&sign=25988815&sv=2) --- # Tensing FME UK Tour 2025 | Avineon Tensing UK - FME + Any AI Training Course [PreviousAvineon Tensing](/fme-+-any-ai-training-course/fme-+-any-ai-training-course/avineon-tensing) [NextAbout this Document](/fme-+-any-ai-training-course/fme-+-any-ai-training-course/about-this-document) Last updated 15 days ago You are very welcome to join us on our FME Tour, if you liked this training, you're going to love this event... it's also free! ### [](#check-out-the-agenda-and-register-to-attend-the-2025-uk-tour-here) These events focus on innovative technical solutions for data wrangling and offers a range of case studies from various fields. Participate in interactive sessions, engage in thoughtful discussions, and connect with professionals who share your interest in technology. * **Learn About New Features:** Discover how recent updates to the FME Platform can improve your GIS and data workflows. Our sessions will provide you with useful knowledge and actionable tips. * **Network with Industry Professionals:** Meet other professionals from the GIS and data community in your region. Exchange valuable experiences and ideas that can enhance your use of FME. * **Evaluate FME's Suitability:** If you're considering FME and wondering if it's right for your needs, this event will provide the insights you require. Talk to Tensing's FME Certified Professionals and understand the full capabilities of the FME Platform. * **Meet Our Technical Experts:** Our team will be on hand to address your questions. Take this opportunity to deepen your knowledge and gain expert advice on leveraging FME effectively. * **Access Exclusive Content:** Learn about the latest developments and upcoming innovations in the FME Platform. * **Have fun while learning:** We believe that learning should be enjoyable! The event is designed to be engaging, allowing you to learn, collaborate, and have a good time. So whether you're a technical user of FME, someone who wants to understand if FME will suit your requirements, or you’re a data specialist looking to broaden your horizons, this event is the perfect opportunity to learn, network, and have some fun! [Check out the agenda and register to attend the 2025 UK tour here!](https://www.tensing.com/en/fme-uk-tour) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fcontent.gitbook.com%2Fcontent%2F3gaZkHJOAvL5gXLEsKl6%2Fblobs%2FTjw4yo3s9XNVE7tDJ4H6%2Fimage.png&width=768&dpr=4&quality=100&sign=30967637&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fcontent.gitbook.com%2Fcontent%2F3gaZkHJOAvL5gXLEsKl6%2Fblobs%2FC7a4wp7Lh200U0qpT5VD%2Fimage.png&width=768&dpr=4&quality=100&sign=f5ab74b5&sv=2) --- # Google Gemini Connectivity | Avineon Tensing UK - FME + Any AI Training Course Artificial Intelligence is also able to help us when it comes to raster data or images. This type of image understanding and extended context processing can help us to solve complex spatial and OCR style data extraction challenges. ### [](#why-choose-gemini) Why Choose Gemini? Gemini offers several unique advantages that make it particularly valuable for GIS and data integration workflows: * Gemini can process text and images together in a single prompt, making it ideal for workflows that involve analysing maps, diagrams, or any visual data alongside textual information. * You can work with more data at once thanks to Gemini's extended context window, which allow for longer prompts and more comprehensive analysis compared to many other AI models. * Gemini Flash provides an economical entry point with impressive capabilities, making advanced AI accessible even with budget constraints. * Gemini shows excellent understanding of maps, diagrams and spatial imagery, making it particularly valuable for GIS professionals. * The models demonstrate strong spatial reasoning capabilities, helping solve complex problems that involve geographical and spatial relationships. ### [](#gemini-models-explained) Gemini Models Explained Model Key Capabilities **Gemini Flash** • Flash delivers remarkably fast response times, making it ideal for workflows where quick processing is important without sacrificing quality. • You'll benefit from significantly lower cost per token compared to larger models, allowing you to process more data within your budget constraints. • The model handles routine tasks and simpler image analysis efficiently, perfect for many day-to-day FME workflow automation needs. • Flash works exceptionally well for basic image classification, simple data extraction, and routine text generation tasks that don't require the full power of larger models. **Gemini Pro** • Pro offers more sophisticated problem-solving capabilities with enhanced reasoning that can handle complex spatial queries and multi-step analysis. • The model manages more complex instructions and nuanced tasks with better context handling, allowing you to create more sophisticated workflows. • You'll receive more precise and detailed responses for complex queries, with higher quality outputs that require less post-processing. • Pro excels at detailed visual analysis and subtle feature detection with improved image understanding that can identify fine details in maps and imagery. ### [](#working-with-extended-context-windows) Working with Extended Context Windows One of Gemini's standout features is its larger context window, which offers significant advantages: * You can include longer text descriptions or multiple images in a single prompt, allowing you to process more data at once and create more comprehensive analysis workflows. * Gemini keeps track of more information without losing the thread, helping maintain context across complex workflows that involve multiple steps or considerations. * The models can process comprehensive reports or lengthy specifications without splitting them, making document analysis more coherent and accurate. * You can present several images simultaneously for comparative analysis, allowing the model to identify patterns, changes, or relationships across multiple visual inputs. For example, you could: * Send an aerial image, parcel boundary data and zoning regulations in a single prompt. * Ask the model to analyse multiple historical maps to identify changes over time. * Include both technical specifications and visual diagrams when requesting analysis. Let's give it a go with a little hands-on practice. [PreviousExercise 1.1 - The OpenAIConnector](/fme-+-any-ai-training-course/chapter-1-ai-foundations-and-fme-integration/openai-connectivity/exercise-1.1-the-openaiconnector) [NextExercise 1.2 - The GoogleGeminiVisionConnector](/fme-+-any-ai-training-course/chapter-1-ai-foundations-and-fme-integration/google-gemini-connectivity/exercise-1.2-the-googlegeminivisionconnector) Last updated 17 days ago --- # OpenAI, Gemini - API Setup | Avineon Tensing UK - FME + Any AI Training Course [PreviousPrompting Techniques](/fme-+-any-ai-training-course/chapter-1-ai-foundations-and-fme-integration/prompting-techniques) [NextOpenAI](/fme-+-any-ai-training-course/chapter-1-ai-foundations-and-fme-integration/openai-gemini-api-setup/openai) Last updated 15 days ago > **Note for training delegates**: Your trainer will provide pre-configured API keys for all of the AI providers that we use in our example FME Workspaces. This information is provided as reference for when you implement these techniques in your own environment. Our example workspaces connect with the OpenAI and Google Gemini services. What follows is an overview of how to set up your own API access, when you're ready to implement these workflows outside of our training environment. ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252Fze14aVBRmliuDdu1IYuH%252Fimage.png%3Falt%3Dmedia%26token%3Df29377dd-b3ac-4c4b-a54f-4e5bd316c861&width=768&dpr=4&quality=100&sign=6f787fc8&sv=2) --- # Course Etiquette | Avineon Tensing UK - FME + Any AI Training Course [PreviousCourse Resources](/fme-+-any-ai-training-course/fme-+-any-ai-training-course/course-resources) [NextChapter 1: AI Foundations & FME Integration](/fme-+-any-ai-training-course/chapter-1-ai-foundations-and-fme-integration) Last updated 15 days ago [](#testing-in-advance) Testing in advance ----------------------------------------------- For online courses, please consider other students and test your virtual machine connection or local FME licence before the course starts. The instructor cannot help debug connection problems or missing course prerequisites components during the course. [](#distraction-audio) Distraction / Audio ----------------------------------------------- For live courses, please respect other students’ needs by keeping noise to a minimum when using a mobile phone or checking e-mail. For online courses, if you have background noise or a separate task to complete, such as to take a call, do please mute your microphone. [](#webcams) Webcams ------------------------- Sharing your webcam for the duration of the course makes a real difference to the overall engagement of the session. From the trainer's perspective we can see from your expression if perhaps we need to explain a technicality in a different way. It also helps everyone on the course to be more engaged...overall, it's just a more satisfying experience and makes things as close to what it would be like if we were all in a room together. That said, we appreciate that especially when working from home, sharing your camera all of the time can sometimes be challenging. We respect your privacy, but if you can share your camera for the training duration, we'd be very grateful! [https://www.microsoft.com/en-gb/](https://www.microsoft.com/en-gb/) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fcontent.gitbook.com%2Fcontent%2F3gaZkHJOAvL5gXLEsKl6%2Fblobs%2Fguv2ZpUvD58yNVLn8bAb%2Fimage.png&width=768&dpr=4&quality=100&sign=8f5b86ef&sv=2) --- # Chapter 2: Building AI-Powered Workflows in FME | Avineon Tensing UK - FME + Any AI Training Course [PreviousExercise 1.2 - The GoogleGeminiVisionConnector](/fme-+-any-ai-training-course/chapter-1-ai-foundations-and-fme-integration/google-gemini-connectivity/exercise-1.2-the-googlegeminivisionconnector) [NextThe HTTPCaller](/fme-+-any-ai-training-course/chapter-2-building-ai-powered-workflows-in-fme/the-httpcaller) Last updated 16 days ago [](#an-introduction-to-apis) An Introduction To APIs --------------------------------------------------------- To interact with an AI service we first need to understand a little about what we call an Application Programming Interface, or API. Picture it as a set of well-defined guidelines and protocols that let different bits of software talk to each other and swap information. Rather than needing to get your head around the fiddly inner workings of another application, you can use its API to ask for specific bits of data, or set off certain functions in a standard manner. An API makes building software more modular and efficient, allowing various systems to work together like clockwork. Now, it's worth bearing in mind that clever artificial intelligence services, such as Google Gemini, also operate using APIs. Just like any other bit of software, Gemini offers specific endpoints that allow developers and other applications to interact with it's capabilities. For instance, one of it's main endpoints lets you add some text and get insightful responses back, having a proper conversation or getting information on all sorts of things. We've already started to see some of these GenerativeAI capabilities in the earlier chapter. Another endpoint allows you to get the gist of lengthy documents in a more manageable form. These endpoints act like doorways, providing a structured way to access the rather powerful AI capabilities from different providers. When we're talking about web-based APIs, a common approach is something called **REST**, or Representational State Transfer. With REST, we typically use different types of requests to carry out various actions. * The `GET` request, for example, is used to retrieve information, think of it as asking politely to see something. * Then you've got `POST`, which is often used to send new data to the server, like submitting a form. * If you need to update existing information, you'd likely use `PUT`. * And finally, `DELETE` does exactly what it says on the tin, it's used to remove resources. These different request types ensure a clear and standardised way for applications to interact with each other over the internet. ### [](#popular-api-documentation) Popular API Documentation The following are some of the popular AI services and their associated API documents: * **Gemini API:** Google's official developer documentation - . * Integrates AI deeply across its vast ecosystem, from search and cloud services to hardware. Gemini is designed to be a multimodal model from the ground up, excelling at understanding and generating across different types of information like text, images, audio, video, and code. Google emphasises its models' ability to handle very large context windows and their seamless integration with Google Cloud and other platforms. * **OpenAI API:** Official OpenAI developer platform documentation - . * Known for its rapid innovation and the widespread popularity of models like GPT and DALL-E. They have a broad focus, pushing the boundaries of AI capabilities in various domains, including text generation, image creation and more recently, multimodal applications with GPT-4o. OpenAI has been a key driver in making advanced AI accessible through its API and user-friendly interfaces like ChatGPT. * **Anthropic API (for Claude):** Anthropic's official documentation for their Claude models - . * Anthropic: Places a strong emphasis on AI safety and ethics. Their development process, particularly with their "Constitutional AI" framework, aims to create AI systems that are helpful, harmless, and honest. They are known for producing models like Claude, which often exhibit strong reasoning and natural language capabilities with a focus on safety. [](#test-an-api) Test an API --------------------------------- API's can be fun, there are also lots of free API services out on the internet that you can make test request to and get to understand how they respond. If you're based in the UK and you want to find out where a specific Postcode is in the country, this is a really useful and free service. In recent years the space race has been reignited, coincidentally by some of the same people behind some of the AI tools we're discussing. So, if you'd like to know who's in space right now, or where the International Space Station is, Open Notify is here to help: * * * ### [](#next-steps) Next Steps This chapter is broken into three sections. The first two sections explore how we prepare the request to the AI service followed by breaking down the response. The third section then takes our learnings from the first two sections to enable us to take a deep dive on what could potentially be the inside an AI customer transformer that you might build. In the next section we'll explain how you can easily use FME to build customised URLs and make REST requests automatically to a specific API endpoint like this : ...and, if numbers are more your thing and you'd like some random number facts, is the corner of the internet you've been searching for. This API provides interesting facts about numbers. You can make a `GET` request to `http://numbersapi.com/{number}` (replace `{number}` with an actual number) to get a fact about that number in plain text. For example, `http://numbersapi.com/42`. You can also try `http://numbersapi.com/random/math` for a random math fact. In this hands-on chapter, we’ll explore how to use FME Transformers to prepare effective requests to AI services via their API and post-process their responses. By combining the power of introduced in the earlier chapter, with targeted data preparation and response handling, you’ll gain a deeper understanding of how to integrate AI into your workflows. [https://ai.google.dev/gemini-api/docs](https://ai.google.dev/gemini-api/docs) [https://platform.openai.com/docs/overview/](https://platform.openai.com/docs/overview/) [https://docs.anthropic.com/en/docs/welcome](https://docs.anthropic.com/en/docs/welcome) [https://api.postcodes.io/postcodes/cb43lt](https://api.postcodes.io/postcodes/cb43lt) [http://numbersapi.com/](http://numbersapi.com/) [prompt engineering](/fme-+-any-ai-training-course/chapter-1-ai-foundations-and-fme-integration/prompting-techniques) [Postcodes.io](https://postcodes.io/) ![Logo](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fpostcodes.io%2Ffavicon.svg&width=20&dpr=4&quality=100&sign=eefb1238&sv=2) [Open Notify -- API Doc | People In Space Now](http://open-notify.org/Open-Notify-API/People-In-Space/) [Numbers API](http://numbersapi.com/#42) --- # Google Gemini | Avineon Tensing UK - FME + Any AI Training Course [PreviousOpenAI](/fme-+-any-ai-training-course/chapter-1-ai-foundations-and-fme-integration/openai-gemini-api-setup/openai) [NextOpenAI Connectivity](/fme-+-any-ai-training-course/chapter-1-ai-foundations-and-fme-integration/openai-connectivity) Last updated 17 days ago Google Gemini refers to a family of multimodal large language models developed by Google DeepMind. Unlike earlier models, Gemini was engineered from the outset to understand and process various forms of information simultaneously, including text, images, audio, video and code. The initial release in December 2023 included different versions like Gemini Ultra (for complex tasks), Gemini Pro (for a wide range of tasks), and Gemini Nano (for on-device applications). These models power Google's AI chatbot, also named Gemini (formerly Bard) and are designed to be a powerful competitor in the generative AI landscape, focusing on advanced reasoning and a broad understanding of different data types. ### [](#getting-started-with-google-gemini) Getting started with Google Gemini: * Register for a Google account if you don't already have one * Visit Google AI Studio * You'll need a credit card on file to use the API on sensitive data through the Google Cloud billing account console * Generate your API key from the AI Studio interface * Review their pricing page for current rates Google offers a generous free tier, making it great for getting started. **Keep in mind that using the free API option means your data may contribute to Google's model training**. For data-sensitive applications, we recommend consider their paid API service. Both services provide excellent documentation to help you integrate these AI capabilities into your FME workflows outside of the training environment. Google AI Studio [https://aistudio.google.com](https://aistudio.google.com/) [https://console.cloud.google.com/billing](https://console.cloud.google.com/billing) [https://ai.google.dev/pricing](https://ai.google.dev/pricing?) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fcontent.gitbook.com%2Fcontent%2F3gaZkHJOAvL5gXLEsKl6%2Fblobs%2F0yxP3GA6i1vWAFjKKvbd%2Fimage.png&width=768&dpr=4&quality=100&sign=44431dd1&sv=2) --- # OpenAI | Avineon Tensing UK - FME + Any AI Training Course [PreviousOpenAI, Gemini - API Setup](/fme-+-any-ai-training-course/chapter-1-ai-foundations-and-fme-integration/openai-gemini-api-setup) [NextGoogle Gemini](/fme-+-any-ai-training-course/chapter-1-ai-foundations-and-fme-integration/openai-gemini-api-setup/google-gemini) Last updated 17 days ago ChatGPT refers to a family of large language models developed by OpenAI, designed to understand and generate human-like text. Originally based on the GPT (Generative Pre-trained Transformer) architecture, ChatGPT has evolved to handle increasingly complex tasks, including conversation, content creation, summarisation, coding assistance, and more. Its capabilities were further expanded with the introduction of GPT-4, which brought improvements in reasoning, creativity and handling nuanced instructions. ChatGPT powers a variety of applications, including OpenAI's own ChatGPT platform and integrations into tools like Microsoft’s Copilot. It continues to play a major role in the generative AI landscape, focusing on natural language understanding, helpfulness and broad accessibility. ### [](#getting-started-with-openai) Getting started with OpenAI: * Visit the OpenAI platform website . * Register with a credit card and setup a billing plan * Access your API keys from the menu * Monitor your usage through their dashboard * Check current pricing on their pricing page OpenAI operates on a consumption-based model where you're charged based on the specific models and token volumes you use. You can monitor your usage via their usage page with pricing found here [https://platform.openai.com/usage](https://platform.openai.com/usage) [https://openai.com/api/pricing/](https://openai.com/api/pricing/) [https://platform.openai.com/](https://platform.openai.com/) [https://openai.com/api/pricing/](https://openai.com/api/pricing/) OpenAI Billing Dashboard ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fcontent.gitbook.com%2Fcontent%2F3gaZkHJOAvL5gXLEsKl6%2Fblobs%2F4EvjPffZi0apM4QOUn2C%2Fimage.png&width=768&dpr=4&quality=100&sign=f5c0b7e9&sv=2) --- # Exercise 1.1 - The OpenAIConnector | Avineon Tensing UK - FME + Any AI Training Course It's time to make a live connection to an AI service, to see if we can return something useful that perhaps may help make your future workspaces go with a little more of a zing! The OpenAIConnector transformer brings the power of large language models right into your FME workflows, opening up exciting possibilities for automating complex text processing that would otherwise require custom coding or manual work. **Remember that in your training environment, API keys have been provided by your trainer.** **Scenario** This workspace allows us to get to grips with a simple request to OpenAI. **Goals** Put your new understanding of some of the myriad AI terminology into practice, so you can learn the difference between a model and a temperature, a prompt and an action! **Start Workspace** C:\\TrainingData\\Workspaces\\FMEAnyAI\\Exercise1.1-Begin.fmw **End Workspace** C:\\TrainingData\\Workspaces\\FMEAnyAI\\Exercise1.1-Complete.fmw **Data** N/A ### [](#id-1-testing-the-openaiconnector) 1) Testing the OpenAIConnector First open FME Workbench and launch the Workspace. We're going to keep things simple with a quick prompt against the text generation action to help set the scene and allow you to see some of the capabilities of AI. Explore all the parameters currently defined in the transformer and run the workspace in it's unedited condition. Your friendly OpenAI service should reply with something along the lines of: > "Hello! How can I assist you today?" ### [](#id-2-now-experiment-with-the-prompt) 2) Now experiment with the Prompt Try replacing the simple prompt with something a little more detailed: Copy Can you tell me a little about the FME Platform from Safe Software? Respond in British English please. It might be interesting for you to see if a simple change in phrasing can lead to a different response. ### [](#id-3-now-experiment-with-adding-some-instructions) 3) Now experiment with adding some Instructions Try adding the following statement to the Instructions to see how the response is adjusted to meet your new demand. Copy Your response needs to appeal to a teenager. As you can see by the different response, an AI can really use the context that you give it to adjust how it responds, not just what it responds with! ### [](#id-4-dynamic-prompts-with-fme-attributes) 4) Dynamic Prompts with FME Attributes To make things more interesting, in our prototype workspace right-click on the 'Created' port of the Creator Transformer. Add an Attribute and set the Name to **Desc** and set the Value to: Copy Presenting a well-maintained semi-detached residence, offering approximately 850 square feet of internal accommodation. Constructed in 1958, this property features three bedrooms and exemplifies a traditional post-war architectural style prevalent throughout the area. An excellent opportunity for discerning buyers seeking a established home. Go back to the OpenAIConnector and ensure your Instruction is deleted! We are done communicating with our teenager. Now click the drop down arrow to the right of the User Prompt, select the option to 'Open Text Editor' and set the prompt to: Copy Extract from this property description @Value(Desc) the year the building was built and the number of bedrooms it has. Return ONLY a valid JSON object with these keys: - bedrooms (an integer value only) - year_built (a 4 digit year) You might, if the wind is blowing in the right direction... get a response like: > { "bedrooms": 3, "year\_built": 1958 } However, don't be surprised if that's not the case. Working with AI is not always an exact science and you may have to be much more constrained with the type of prompt you supply, or even change models or suppliers completely. Either way... just imagine if you had thousands of rows of unstructured data that you wanted to structure in some way like this. Note that that we were really specific about what format we wanted the response back in and how that impacted the structure of the response. This gives us a much better chance of being able to parse the response further along in the Workspace. We could also go one step further and provide some example responses, as this will help the AI to do an even better job. * * * If your trainer is able to give you a few minutes to be creative, why not try letting your imagination run free... but not too free, we have a lot to get through! * * * ### [](#some-further-things-to-consider) Some further things to consider: #### [](#best-practices) Best Practices * Consider using multiple transformers in sequence for complex tasks. * Monitor AI token usage to manage costs. * Cache responses with FME when appropriate, to avoid unnecessary API calls. #### [](#common-challenges) Common Challenges * **Token limits**: For OpenAI models the context window isn't that big, keep an eye on the length of the prompt. * **Response formatting**: Use explicit instructions about output format in your prompt. For example, if you prefer a British English response, rather than the more likely American English response, make that clear in your request! * **Cost management**: Monitor usage and consider batching requests when possible. OpenAI and Azure OpenAI are now some of the more expensive models on the market, so keep an eye on usage. [PreviousOpenAI Connectivity](/fme-+-any-ai-training-course/chapter-1-ai-foundations-and-fme-integration/openai-connectivity) [NextGoogle Gemini Connectivity](/fme-+-any-ai-training-course/chapter-1-ai-foundations-and-fme-integration/google-gemini-connectivity) Last updated 15 days ago ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252FAnNY1E7zfXgbLVvPDhBM%252Fimage.png%3Falt%3Dmedia%26token%3Dad55a189-b69a-45cf-a1a6-a7dceaa010e8&width=768&dpr=4&quality=100&sign=178ce205&sv=2) --- # Create a Roboflow Account | Avineon Tensing UK - FME + Any AI Training Course [PreviousChapter 4: AI-Driven Image Processing with Roboflow](/fme-+-any-ai-training-course/chapter-4-ai-driven-image-processing-with-roboflow) [NextCreate Your First Project](/fme-+-any-ai-training-course/chapter-4-ai-driven-image-processing-with-roboflow/create-your-first-project) Last updated 16 days ago First things first, you need a Roboflow account. **1)** Navigate to the . **2)** Sign up using your email, Google account, or GitHub account. **3)** In 'Name Your Workspace' enter: "Training". [Roboflow website](https://roboflow.com/) --- # The Power of Custom Transfomers | Avineon Tensing UK - FME + Any AI Training Course [PreviousExercise 2.2 - Parsing a JSON Response](/fme-+-any-ai-training-course/chapter-2-building-ai-powered-workflows-in-fme/understanding-json/exercise-2.2-parsing-a-json-response) [NextExercise 2.3: Exploring an AI transformer](/fme-+-any-ai-training-course/chapter-2-building-ai-powered-workflows-in-fme/the-power-of-custom-transfomers/exercise-2.3-exploring-an-ai-transformer) Last updated 16 days ago [](#what-is-a-custom-transformer) What is a Custom Transformer? -------------------------------------------------------------------- A Custom Transformer is a sequence of standard Transformers condensed into a single Transformer. Any existing sequence of Transformers can be turned into a Custom Transformer. Custom Transformers can be shared by users. For example, our LocalGenerativeAICaller Custom Transformer is shared via the . We don't allow access to the components inside because that makes it harder for us to support it, but when we unsecure it this is what the transformer looks like on the canvas. A custom transformer is basically a workspace disguised and with a much more user friendly interface, like this: [](#custom-transformer-purposes) Custom Transformer Purposes ----------------------------------------------------------------- Among other functions, Custom Transformers help to: * Reuse Content * A sequence of Transformers encapsulated in a single object can be reused throughout a Workspace and shared with colleagues * Employ advanced functionality * Using a Custom Transformer enables additional advanced functionality to be deployed, such as _**looping**_ and/or **parallel processing** * It's one way you can keep your Workspaces tidy (although collapsible bookmarks, are perhaps a more practical way to do that these days!) * By condensing chunks of content, the Workspace canvas becomes less cluttered [](#custom-transformers-and-apis) Custom Transformers and APIs ------------------------------------------------------------------- Interacting with APIs can clutter workspaces, and since specific APIs are often used across multiple workspaces, creating a custom transformer offers a logical solution for better organisation and reusability. In the following exercise we're going to take a look at a workspace that ultimately, could be wrapped up into a custom transformer. Whilst we won't have time to take it that far in this course, we do cover Custom Transformers in much more detail on our FME Form Advanced training course, so please ask your trainer about it if you're interested. [](#custom-transformers-from-avineon-tensing) Custom Transformers from Avineon Tensing ------------------------------------------------------------------------------------------- We're proud to be an approved and 'Verified' Safe Software FME Hub Publisher and many of the custom transformers that we're released are focused on interacting with different AI services. You can access them here: [FME Hub](https://hub.safe.com/publishers/tensing) [FME Hub](https://hub.safe.com/publishers/tensing) ![Logo](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fhub.safe.com%2Ffavicon.ico&width=20&dpr=4&quality=100&sign=e8d2b584&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252FGDzRoUOF47P0ZmpBrKm2%252Fimage.png%3Falt%3Dmedia%26token%3D44619cf8-a5b7-49dc-808d-0a6622e860ce&width=768&dpr=4&quality=100&sign=4d435519&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252FxYCkDCRchQ7HddYV6p1q%252Fimage.png%3Falt%3Dmedia%26token%3D57ed13f9-a0b3-454e-98d8-847830156d8c&width=768&dpr=4&quality=100&sign=ce05361f&sv=2) --- # Upload Your Data | Avineon Tensing UK - FME + Any AI Training Course [PreviousCreate Your First Project](/fme-+-any-ai-training-course/chapter-4-ai-driven-image-processing-with-roboflow/create-your-first-project) [NextAnnotate Your Data (Labeling)](/fme-+-any-ai-training-course/chapter-4-ai-driven-image-processing-with-roboflow/annotate-your-data-labeling) Last updated 16 days ago Now, let's add the images to your project. **1)** You'll be taken to the **Upload** page. **2)** Methods for Uploading: * **Drag and Drop:** Simply drag your image files or folders directly onto the upload area. * **Select Folder\\Files:** Use the file browser to select your data. You will find the training images under: `C:\TrainingData\FMEAICourse\Day_1_Chapter 4\Training Datasets` Roboflow accepts various formats (JPG, PNG, BMP, videos like MP4, MOV, etc.). We won't be using videos but as an FYI - if you upload a video, Roboflow can help you sample frames from it. **3)** Select around 50 images. **4)** Click "**Save and Continue**" and wait for the files to process. It can take a few minutes to upload the images, depending on your internet speed. Once the images are uploaded, jump to the next section. ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252FQpGmSp8kTaammvJ4KuCb%252Fimage.png%3Falt%3Dmedia%26token%3Db640e46f-f7ca-4bc6-9aa3-95bd5a586971&width=768&dpr=4&quality=100&sign=f947214e&sv=2) --- # Create Your First Project | Avineon Tensing UK - FME + Any AI Training Course [PreviousCreate a Roboflow Account](/fme-+-any-ai-training-course/chapter-4-ai-driven-image-processing-with-roboflow/create-a-roboflow-account) [NextUpload Your Data](/fme-+-any-ai-training-course/chapter-4-ai-driven-image-processing-with-roboflow/upload-your-data) Last updated 16 days ago Each project corresponds to a specific computer vision problem you want to solve. **1)** Click the "**Create New Project**" - Click 'Projects' on the left hand side panel, then '+ New Project'. **2)** We have three sets of image classifications for you to choose from Penguins, Tables in PDFs and Car Number Plates. **3) Configure Your Project:** * **Project Name:** Name the project, depending on which dataset you want to use name the project accordingly e.g. Penguins, Tables in PDFs or Number Plates. * **Project Type:** This is crucial! We are using **Object Detection** - we would like bounding boxes around objects you want to find (e.g., detecting penguins, tables or number plates). * **Annotation Group:** Enter the names of the classes (labels) you'll be using (e.g., "penguin", "table", "number-plate"). * **Visibility:** We are using some open data, so in this case lets select 'Public Domain'. **4)** Click "**Create Public Project**". ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252FTU3g2O8G4Lh8X0FSPFAd%252Fimage.png%3Falt%3Dmedia%26token%3Dffd1d169-c856-4688-a57f-766e45809289&width=768&dpr=4&quality=100&sign=9e21ed67&sv=2) --- # Exercise 3.1 - Local Hello World | Avineon Tensing UK - FME + Any AI Training Course [PreviousChapter 3: Local AI](/fme-+-any-ai-training-course/chapter-3-local-ai) [NextExercise 3.2 - GDPR Checker](/fme-+-any-ai-training-course/chapter-3-local-ai/exercise-3.2-gdpr-checker) Last updated 16 days ago Let's get started. Why not create a small "Hello World" process, but with a slight twist. Let's ask for the output in JSON. As we have already seen today if we can output structured JSON, FME loves JSON and we can really go to town! **Scenario** Configure an integration with a locally deployed AI service and ask a simple question. **Goals** Highlights one of the powers of Generative AI - unstructured data in, structured data out. **Start Workspace** C:\\TrainingData\\Workspaces\\FMEAnyAI\\Exercise3.1-Complete.fmw **End Workspace** N/A **Data** None ### [](#id-1-open-and-run-the-starting-workspace) **1) Open and Run the Starting Workspace** For this Workspace to run correctly, we will need to ensure that the gemma2:2b model from Ollama's library of models is installed. We can do this by opening a command prompt / PowerShell in your Virtual Machine (VM) and type the following: Copy ollama list Upon pressing enter we should see the gemma2:2b model listed. If you want to download additional models you can use the following command (where mistral is the name of the model): `ollama pull mistral` Within this example we are asking how the colour of the sky changes during the day. The output will be in JSON format. **Note about performance** - you will notice fairly quickly the performance is not super quick. We are using a fairly small virtual environment. If you were to run this locally on your own computer, your performance will be better, even if you are using a machine that is a couple of years old. If you are lucky enough to have a dedicated graphics card, this is going to fly! This Workspace uses the Large Language Model (LLM) to run the model locally through the Avineon Tensing made FME Hub Transformer. For a complete list of available models go to [Google Gemma2](https://ollama.com/library/gemma2:9b) [LocalGenerativeIAICaller](https://hub.safe.com/publishers/tensing/transformers/localgenerativeaicaller) [https://ollama.com/library](https://ollama.com/library) FME: Making order from chaos since 1993! Hello sky example with output in JSON ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252FKFQ0hgZgA1GVdpk5FVJR%252Fimage.png%3Falt%3Dmedia%26token%3Dc770d23e-78d5-446b-9b50-d2461cd34117&width=768&dpr=4&quality=100&sign=1edaddcf&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fcontent.gitbook.com%2Fcontent%2F3gaZkHJOAvL5gXLEsKl6%2Fblobs%2FCADghFqDYYq39uER69aD%2Fimage.png&width=768&dpr=4&quality=100&sign=a5ddb63c&sv=2) --- # Chapter 3: Local AI | Avineon Tensing UK - FME + Any AI Training Course [PreviousExercise 2.3: Exploring an AI transformer](/fme-+-any-ai-training-course/chapter-2-building-ai-powered-workflows-in-fme/the-power-of-custom-transfomers/exercise-2.3-exploring-an-ai-transformer) [NextExercise 3.1 - Local Hello World](/fme-+-any-ai-training-course/chapter-3-local-ai/exercise-3.1-local-hello-world) Last updated 16 days ago [](#ollama-and-open-source-models) Ollama and Open Source Models --------------------------------------------------------------------- Generative AI has been everywhere recently and through some of the example uses we have already seen, it can be incredibly powerful. However, there are some worries about using third-party services and rightly so. **Security, Cost, Data Protection, third-party dependencies**, to name but a few, are all areas of consideration when thinking about incorporating Generative AI into a data workflow/pipeline within your organisation. We know, through our experience, that some of our customers have locked down their environments to AI completely at the moment. On the other hand some have adopted a more private platform approach to AI tooling, through their own platform such as Microsoft Azure... whilst others are all in. The choice is going to have to be personal. There is another option though that we haven't discussed yet. Fortunately, there are Open Source Large Language Models (LLMs) available under permissible 'commercial use' licenses and there are also ways to run these models locally - as in, on your laptop, or one of your own servers without calling outside to the external internet. This takes away one of the primary areas of perceived risk that your organisation's IT team may cite as a reason to avoid AI. Using it is possible to run generative AI models locally on Windows, Mac and Linux. Ollama.com run large language models locally ### [](#running-ai-locally-with-ollama) Running AI locally with Ollama But how do we connect to it? Well, you can do so via the command line (as you can see below), Ollama allows you to install a range of models, such as Deepseek, Gemma (built on Google's Gemini technology) or Mistral with simple commands such as "ollama pull mistral". Ollama provides a straightforward API that lets you integrate these powerful models into your applications and scripts. With a simple HTTP request to localhost:11434, you can send prompts and receive responses just as you would with cloud-based services. This API-based approach means you can connect your FME workflows to Ollama using the HTTPCaller transformer, bringing local AI capabilities right into your data processing orchestrations! Alternatively, you can use the FME Hub Transformer we made - the LocalGenerativeAICaller and that means you don't need to configure the request or parse the response yourself. ### [](#open-source-advantages) Open Source Advantages Open source models are a game-changer for many of our users working with sensitive data. Unlike their proprietary cousins, open source models share their code, weights and training approach with everyone. Some of these advantages might be: * Your data stays put on your machine - no travelling to external services! * No worries about your information being used to train someone else's AI. * Freedom to customise models for GIS terminology or your industry's unique vocabulary. * Perfect for those secure environments where internet access can be difficult or your bandwidth is poor. * ...and importantly, your finance department will love the cost savings compared to pay-per-token cloud services. While these open models might not yet match the biggest proprietary systems in every capability (rather like comparing a dependable hatchback to a luxury sports car), they're developing at an impressive pace. For your FME workflows, this means more options for bringing AI into your data processing whilst keeping your IT security team happy and your budget intact! * * * Check out the following examples to explore some of the possibilities. [Ollama](https://ollama.com/) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Flh7-us.googleusercontent.com%2Fslidesz%2FAGV_vUcIDn5CAbbaUFJ2PTfEtHlZWRiWE9AwhWYAqcowiQDAOhSZgSnhrRyQJlsx3OzULJJ2cyHtD4Q94a2VLawZsW3jQFLiTtIZAFtYOGYKOwfF4PnGdsJ3Cxp4VOFzZpJElqOiD0vDB86P5cL4S9zHSd-yei0wKTRL%3Ds2048%3Fkey%3DX3ziQmhsNYxl1FnSgEMUYA&width=768&dpr=4&quality=100&sign=a03e567e&sv=2) [FME Hub](https://hub.safe.com/publishers/tensing/transformers/localgenerativeaicaller) ![Logo](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fhub.safe.com%2Ffavicon.ico&width=20&dpr=4&quality=100&sign=e8d2b584&sv=2) HTTPCaller configured for using with Ollama ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252Fh2saFkTDsqMqGiog8nMF%252Fimage.png%3Falt%3Dmedia%26token%3D2f98495f-dc92-4b15-9274-b0a08f375cf4&width=768&dpr=4&quality=100&sign=78fbc755&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fcontent.gitbook.com%2Fcontent%2F3gaZkHJOAvL5gXLEsKl6%2Fblobs%2F8Dv5nhNkLZPVcNrHT6pV%2Fhi%2520ollama.gif&width=768&dpr=4&quality=100&sign=f5164860&sv=2) --- # Exercise 2.2 - Parsing a JSON Response | Avineon Tensing UK - FME + Any AI Training Course **Scenario** This workspace allows us to take the NHL example one step further and shows you how we would parse the JSON response that has been provided by Gemini AI. **Goals** Test out your understanding of JSON structures and try some of the key tools for manipulating it. **Start Workspace** C:\\TrainingData\\Workspaces\\FMEAnyAI\\Exercise2.2-Begin.fmw **End Workspace** C:\\TrainingData\\Workspaces\\FMEAnyAI\\Exercise2.2-Complete.fmw **Data** NHL\_Teams\_2024-25 (CSV) ### [](#deciphering-the-response-from-the-google-gemini-api) Deciphering the response from the Google Gemini API This is a continuation of the previous HTTPCaller exercise. We shall look at the JSON response and investigate a technique to extract the information we need and create suitable attributes from the AI derived NHL Team facts. ### [](#id-1-with-fme-workbench-open) **1) With FME Workbench Open** * Continue with your workspace from the last exercise or if that isn't available, open Exercise2.2-Begin.fmw ### [](#id-2-add-a-jsonextractor) **2) Add a JSONExtractor** * Drop a JSONExtractor into your workspace canvas. This transformer will extract the structured JSON Ouput that was generated by Google Gemini. * Connect it to the HTTPCaller * Double click on the JSONExtractor to open the Parameters dialog * Set Input Source to: JSON Document * Set JSON Document to use the '\_response body' attribute * Set Target Attribute to: TeamDetailsJSON * Set JSON Query to: Copy json["candidates"][*]["content"]["parts"][*]["text"] ### [](#id-3-add-a-jsonflattener) **3) Add a JSONFlattener** * Drop a JSONFlattener into your workspace. This transformer will create attributes out of the keys in the extracted JSON (TeamDetailsJSON). * Connect it to the JSONExtractor * Double click on the JSONFlattener to open the Parameters dialog * Set Input Source to: JSON Document * Set JSON Document to use the 'TeamDetailsJSON' attribute * Set Attributes to Expose to: * Year Founded * Arena Name * Stanley Cups ### [](#id-4-clean-up-the-results) **4) Clean up the Results** * Run the workspace in it's current state in cache mode. * Inspect the cached results out of the JSONExtractor, then compare that to the output from the JSONFlattener. * It needs a little work just to tidy things up. Add an AttributeManager and a Sorter connected in series, straight after the JSONFlattener. * In the AttributeManager, remove any attribute that is not originally from the CSV or that you have successfully parsed from Gemini. You should be left with: * Team * Division * Conference * Year Founded * Arena Name * Stanley Cups * With the Sorter, set it to 'Sort By' the 'Stanley Cups' attribute and use a Numeric/Descending sort. That way you can see the most to the least successful NHL teams. At this point, if our Certified Trainer Simon Green is delivering your training, now is perhaps a good time to ask him who he supports! Assuming he hasn't spent the last hour indoctrinating you on the NHL of course! Here's a clue: Well done, you can now consider yourself thoroughly introduced to the world of APIs and JSON. * * * ### [](#taking-it-further) Taking it further Here are some resources to further your understanding of the JSON format and how to work with it in FME. [PreviousUnderstanding JSON](/fme-+-any-ai-training-course/chapter-2-building-ai-powered-workflows-in-fme/understanding-json) [NextThe Power of Custom Transfomers](/fme-+-any-ai-training-course/chapter-2-building-ai-powered-workflows-in-fme/the-power-of-custom-transfomers) Last updated 16 days ago Introduction to the JSON Format: . JSON Tutorial: . JSON Reading: . Advanced JSON Reading: . JSONExtractor, JSONFlattener & JSON Fragmenter: . [https://www.w3schools.com/js/js\_json\_intro.asp](https://www.w3schools.com/js/js_json_intro.asp) [https://support.safe.com/hc/en-us/articles/25407515641997-Tutorial-Getting-Started-with-JSON](https://support.safe.com/hc/en-us/articles/25407515641997-Tutorial-Getting-Started-with-JSON) [https://support.safe.com/hc/en-us/articles/25407533512077-Reading-JSON](https://support.safe.com/hc/en-us/articles/25407533512077-Reading-JSON) [https://support.safe.com/hc/en-us/articles/25407778152717-Advanced-JSON-Reading](https://support.safe.com/hc/en-us/articles/25407778152717-Advanced-JSON-Reading) [https://support.safe.com/hc/en-us/articles/25407515351949-Transforming-JSON-using-the-JSONExtractor-JSONFlattener-and-JSONFragmenter](https://support.safe.com/hc/en-us/articles/25407515351949-Transforming-JSON-using-the-JSONExtractor-JSONFlattener-and-JSONFragmenter) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252FaUcVTAkq1fhQmEWQiGHn%252FSimon-Green-hobby.jpg%3Falt%3Dmedia%26token%3D2ac78ae7-b97a-4f59-854c-259fcf79f673&width=768&dpr=4&quality=100&sign=ce020437&sv=2) --- # Exercise 3.4 - Sentiment Analysis | Avineon Tensing UK - FME + Any AI Training Course [PreviousExercise 3.3 - Local AI Translator](/fme-+-any-ai-training-course/chapter-3-local-ai/exercise-3.3-local-ai-translator) [NextChapter 4: AI-Driven Image Processing with Roboflow](/fme-+-any-ai-training-course/chapter-4-ai-driven-image-processing-with-roboflow) Last updated 16 days ago In this particular example, the Workspace is combining output from a cloud based generative language model and using this as input into a local generative AI. The basis of this process is to analyse the sentiment in complaints from local residents about overnight roadworks and produce an appropriate written response from the company responsible for the works. **Scenario** Some overnight roadworks need to be carried out on a major road, so we need to find residents nearby that might be impacted by the noise, smell and light and contact them. **Goals** Understand a more involved process that incorporates some spatial analysis and an interaction with a traditional API and learn how chaining AI services can give you flexibility whilst ensuring security at the same time. **Start Workspace** C:\\TrainingData\\Workspaces\\FMEAnyAI\\Exercise3.4-Complete.fmw **End Workspace** N/A **Data** Complaints (CSV) This example workspace reads a CSV file of complaints and geocodes residents' addresses by postcode. The complaint is then sent to OpenAI via the OpenAIChatGPTConnector for analysis, identifying the complainant's demand, type, and severity. Complaints are spatially filtered to determine whether the complainants are near the works. Any complaints that are directly impacted are sent to a locally running AI model, which generates a written response for the construction company to send to the complainant. Within this example, we make use of a list attribute to extract address information and the JSONFlattener to handle the response from the locally running AI and create attributes. We should perhaps say now that this data is completely fictitious and any similarity to any real people, or their actual address is a one in a million coincidence! ### [](#id-1-open-and-run-the-starting-workspace) **1) Open and Run the Starting Workspace** The process starts with the reading of the complaints data and extracts the postcode from the complainant's address. A unique prompt required for the generative AI’s initial categorisation of the complaint is then created and passed to the OpenAIChatGPTConnector custom transformer for analysis. The complaint is assessed for proximity to the engineering works, and if the complainant is impacted, it is sent to the LocalGenerativeAICaller to generate a suitable response. If the complainant is not near the works, a generic response is provided instead. ### [](#id-2-general-sentiment) **2) General Sentiment** The first part of this workspace looks at reading in the complaints we have received, finding an address and location for where the complaint has come from and then creating an initial prompt to send to OpenAI to determine the complaint type, severity and demands. Copy { "Guidance": { "complaint": "This is a complaint that has been submitted: @Value(Complaint).", "complaintDemand": "Does the complainant demand anything such as change of schedule, compensation, reduced noise, reduced smell, reduced light or a complete stop. If there is no demand, set the value to none.", "complaintSeverity": "Is the complaint citing problems with respiratory impact, sleep deprivation, mental health or discomfort or unknown.", "complaintType": "Is the complaint related to noise, smell, light or other? Respond with only those allowed types.", "context": "I am the engineering contractor 'Eagle Engineering' conducting overnight highway maintenance works on the M6 near Birmingham in the UK. The operations are only for 2 weeks, between December 16th and 27th, we will not work at the weekend. The operations are noisy, we're using heavy machinery which creates noise, we have lighting units that are bright and there will be a smell of tarmac and assorted materials that are used on the highway. Some local residents will be impacted overnight but this cannot be avoided in order to deal with major upgrade works for the infrastructure of this important motorway interchange. A number of residents have complained about the overnight works." }, "Important_Note": "Very important: No comments or descriptions, only return the JSON.", "Instructions": "Please use the Guidance to analyse the overnight highway maintenance works complaint and report on the metrics. Use all the keys ['context', 'complaint', 'complaintType', 'complaintSeverity', 'complaintDemand'] to help formulate the response. The output must be JSON and structured as per the Output_Schema", "Output_Schema": { "complaintDemand": "", "complaintSeverity": "", "complaintType": "" } } ### [](#id-3-extracting-sentiment-and-creating-a-response) 3) Extracting Sentiment and Creating a Response From our JSON output from OpenAI we can discover what sort of action and response we might like to take from the complaints we have received. But having to personally write replies to possibly hundreds of complaints on big engineering works would take hours. Instead we can use our Local AI model to personalise our responses based on the complaint, after we have validated that the person complaining actually lives near the works and is therefore valid. Fun fact, this example has been built on a real life scenario that one of our consultants was involved with. In that case a significant number of the people complaining lived absolutely no where near the overnight works and couldn't possibly be impacted, but the lure of some compensation was a little too enticing! First, using the geocoded postcode, the complaint is assessed for proximity to the engineering works, to ensure that the complainant is impacted from our works. If it passes the SpatialFilter, we send the individual complaint to the LocalGenerativeAICaller to generate a suitable response. If the complainant is not near the works, a generic response is provided instead. From there we could send the responses to a team to review before they get sent out via email or regular post. Either way, we definitely think this process is just a way to generate an initial draft that should be moderated by a human before being issued to a real member of the public. Copy { "Guidance": { "complaint" : "@Value(Complaint)", "severity" : "@Value(complaintSeverity)", "name": "@Value(Name)" }, "Instructions": "Using the Guidance, respond appropriately, acknowledging the details of the complaint, outlining when the works are taking place and that they are only for 2 weeks. Apologise sincerely and confirm it has been escalated for further action. Start the response with a salutation addressing @Value(Name). The output must be JSON and structured as per the Output_Schema", "Important_Note": "Very important: No comments or descriptions, only return the JSON.", "Output_Schema": { "feedbackResponse": "", "fromName": "", "title" : "Community Support Representative", "department" : "Eagle Engineering's Birmingham M6 Operations Team" } } **Note about performance** - you will notice fairly quickly the performance is not super quick. We are using a small virtual environment. If you were to run this locally on your own computer, your performance will be better, even if you are using a machine that is a couple of years old. If you are lucky enough to have a dedicated graphics card, this is going to fly! This Workspace uses to run the Large Language Model (LLM) locally through the Avineon Tensing made FME Hub Transformer. [Ollama](https://ollama.com/) [Google Gemma2](https://ollama.com/library/gemma2:9b) [LocalGenerativeIAICaller](https://hub.safe.com/publishers/tensing/transformers/localgenerativeaicaller) Sentiment analysis of complaints in FME ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252F6cn6BCAJ5PC0alhsLpim%252Fimage.png%3Falt%3Dmedia%26token%3D8b619b05-3d17-4d1e-a6f2-64f10986da1d&width=768&dpr=4&quality=100&sign=f82d7e6f&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252FVjMrF0hourW4zYKkkuHh%252Fimage.png%3Falt%3Dmedia%26token%3D719da700-8948-429c-94b1-631d1696fb8e&width=768&dpr=4&quality=100&sign=e3a3edb1&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fcontent.gitbook.com%2Fcontent%2F3gaZkHJOAvL5gXLEsKl6%2Fblobs%2FdhfnkDuZ7NP8kEMXkebQ%2FScreenshot%25202025-03-03%2520171608.png&width=768&dpr=4&quality=100&sign=2ab99cbb&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fcontent.gitbook.com%2Fcontent%2F3gaZkHJOAvL5gXLEsKl6%2Fblobs%2FeEmXpOkQuhN4y3R8HVuH%2FRoadworks.webp&width=768&dpr=4&quality=100&sign=9caf5071&sv=2) --- # Exercise 2.1 - Mastering the HTTPCaller | Avineon Tensing UK - FME + Any AI Training Course [PreviousThe HTTPCaller](/fme-+-any-ai-training-course/chapter-2-building-ai-powered-workflows-in-fme/the-httpcaller) [NextUnderstanding JSON](/fme-+-any-ai-training-course/chapter-2-building-ai-powered-workflows-in-fme/understanding-json) Last updated 16 days ago **Scenario** This workspace allows us to get to grips with the HTTPCaller. **Goals** Make an API request to the Gemini endpoint and get back a successful response. **Start Workspace** C:\\TrainingData\\Workspaces\\FMEAnyAI\\Exercise2.1-Begin.fmw **End Workspace** C:\\TrainingData\\Workspaces\\FMEAnyAI\\Exercise2.1-Complete.fmw **Data** NHL\_Teams\_2024-25 (CSV) ### [](#calling-the-google-gemini-api) Calling the Google Gemini API Let's dive into a practical example where we'll connect directly to Google's Gemini API. Think of this as peeking under the bonnet of the to see how things work. Imagine we had a CSV file containing information on all 32 teams in the ... for the uninitiated, the National Hockey League is a professional ice hockey league in North America composed of 32 teams – 25 in the United States and 7 in Canada. We want to generate information from Gemini around the history of each of the teams. ### [](#id-1-open-the-starting-workspace) **1) Open the starting Workspace** * The only thing that exists in this starting workspace is a parameter that is configured to receive the Google API key. * Add a CSV reader, pointing to the following dataset: Copy C:\TrainingData\Data\Sport\NHL_Teams_2024-25.csv * Click the Reader Parameters button to confirm the CSV has been correctly parsed. ### [](#id-2-create-your-request-body) **2) Create your request body** * First, we'll need to prepare a JSON request body to use in the JSONTemplater. We've done that here for you, so you can copy it directly: Copy { "contents": [\ {\ "parts": [\ {\ "text": " For each team, provide the year they were founded, the number of Stanley Cups they have won, and the name of their home arena: @Value(Team)"\ }\ ]\ }\ ], "generationConfig": { "maxOutputTokens": 800, "responseMimeType": "application/json", "responseSchema": { "properties": { "Arena Name": { "type": "string" }, "Stanley Cups": { "description": "Number of Stanley Cups as a string of digits", "type": "string" }, "Year Founded": { "description": "Year should be a 4-digit string", "type": "string" } }, "required": [\ "Year Founded",\ "Stanley Cups",\ "Arena Name"\ ], "type": "object" }, "temperature": 0.7 } } * Notice how we've cleverly baked the FME attribute "Team" right into our prompt? This means each feature can ask Gemini something unique! ### [](#id-3-add-a-jsontemplater) **3) Add a JSONTemplater** * Drop a JSONTemplater into your workspace and double click on it to open the Parameters dialog. This transformer will create the JSON payload that will get uploaded with the POST request. * Copy and paste the JSON request body from step 2 into the Template section of the Root Template. * Set the Output Attribute Name 'Result' to: UploadBody. ### [](#id-4-add-an-httpcaller) **4) Add an HTTPCaller** * Drop an HTTPCaller into your workspace and double click on it to open the Parameters dialog. This transformer will make the HTTP POST request to Google Gemini AI. * Use the following as the Request URL: Copy https://generativelanguage.googleapis.com/v1beta/models/gemini-2.0-flash:generateContent * Set HTTP Method to: POST * We're sending data, not just asking for it * Add your API key as a Query String Parameter: * Name: key * Value: Select the parameter that was already defined in the workspace called $(google\_api) * In the Body Section: * Set Upload Data to: Specify Upload Body * Set Upload Body to: UploadBody * Set Content Type to: JSON (application/json) * It's like telling the server which language we're speaking ### [](#id-5-run-the-workspace) **5) Run the Workspace** * The Prompt and Run dialog box will immediately appear because we haven't yet add the key to satifgy the google\_auth parameter, so copy that in now (it'll have been provided by your trainer already). * If the workspace completes without errors we can move on to deciphering the response. * If it fails have a go at debugging yourself, or ask your trainer for help. Whilst we pass the API key as a Query String Parameter in this example for ease, it would be more secure to create a Google Cloud AI OAuth Web Connection and use this instead. This uses OAuth2.0 to authenticate and requires configuration in both FME and Google Cloud. While the purpose-built GoogleGeminiVisionConnector makes life easier, understanding how to make these calls directly with the HTTPCaller is incredibly useful when you need to customise things or when something isn't quite working as expected! * * * ### [](#taking-it-further) Taking It Further The HTTPCaller is the foundation for many of FME's more specialised web connectors: * The OpenAIChatGPTConnector is built on similar principles but optimised for AI interactions * The GoogleGeminiVisionConnector extends these capabilities to visual AI * Many cloud service connectors use the same underlying HTTP communication methods, such as the more standard ArcGISOnlineConnector transformer. By mastering the HTTPCaller, you're building skills that transfer directly to these more specialised tools whilst maintaining the flexibility to connect to virtually any web service out there! Consider the OpenAPICaller instead of the HTTPCaller to make your API requests instead, it allows you to access an API endpoint defined by an OpenAPI specification via HTTP or HTTPS. * * * Now we have configured how to send an HTTP Request let's look at how we might decipher the response in the next section! Create an OAuth client ID: [GoogleGeminiConnector](https://hub.safe.com/publishers/safe-sandbox/transformers/googlegeminiconnector) [NHL](https://www.nhl.com/) [https://cloud.google.com/endpoints/docs/frameworks/java/creating-client-ids](https://cloud.google.com/endpoints/docs/frameworks/java/creating-client-ids) [FME Help for the OpenAPICaller Transformer](https://docs.safe.com/fme/html/FME-Form-Documentation/FME-Transformers/Transformers/openapicaller.htm) --- # Exercise 3.2 - GDPR Checker | Avineon Tensing UK - FME + Any AI Training Course [PreviousExercise 3.1 - Local Hello World](/fme-+-any-ai-training-course/chapter-3-local-ai/exercise-3.1-local-hello-world) [NextExercise 3.3 - Local AI Translator](/fme-+-any-ai-training-course/chapter-3-local-ai/exercise-3.3-local-ai-translator) Last updated 16 days ago The General Data Protection Regulation (Regulation (EU) 2016/679, abbreviated GDPR) is a regulation on information privacy in the European Union (EU) and the European Economic Area (EEA). The GDPR is an important component of EU privacy law and human rights law, in particular Article 8(1) of the Charter of Fundamental Rights of the European Union. Source and more details: . What better way to use a locally running AI tool than to try to analyse PDFs containing information that you would never dream of uploading to a third-party cloud service. That's right, data that might contain personal information about you, your customers or your citizens. **Scenario** Analyse data containing information that you would never dream of uploading to a third-party cloud service. **Goals** Use an AI service to help to identify GDPR related data in some sample data. **Start Workspace** C:\\TrainingData\\Workspaces\\FMEAnyAI\\Exercise3.2-Begin.fmw **End Workspace** C:\\TrainingData\\Workspaces\\FMEAnyAI\\Exercise3.2-Complete.fmw **Data** GDPR associated data (PDF) ### [](#id-1-open-the-starting-workspace) **1) Open the Starting Workspace** This example Workspace loads 10 PDFs, 5 containing GDPR sensitive information and 5 that don't. The LocalGenerativeAICaller here is used to spot the GDPR sensitive PDFs. The process has a few more steps than the previous example because the Workspace extracts text from PDF's before passing it over to the LocalGenerativeAICaller transformer, to spot the GDPR sensitive information. ### [](#id-2-read-in-all-pdf-files-and-text-data) 2) Read in all PDF Files and Text Data This first part of the workspace has already been configured. In the first section we are using a **FeatureReader** to read in the Directory and File Pathnames from our C:\\TrainingData\\Data folder. The Directory and File Pathnames Reader is a really useful tool if you haven't used it before. It reads the metadata about your data, not the data itself, in this case we want the path of any PDF files we find. That's why we have 2 FeatureReaders, first we find the PDFs, then we read them! It's a neat trick. The **TestFilter** is used to ensure we are only keeping the PDF files. The second **FeatureReader** then reads in the text information from each page of our PDF files. We are then adding a **Sampler** to filter off two of the PDF files to read. ### [](#id-3-creating-a-prompt) 3) Creating a Prompt In this next step, you will need to create a prompt to check if the PDF you have sampled contains GDPR sensitive information. Open the **AttributeCreator** in the yellow bookmark. Create an Output Attribute called **prompt** and paste the following into its Value (it's also available in the blue bookmark). Copy Review the following text and report back TRUE if it contains GDPR related information. Reply back in JSON. ###Example Output {"gdpr": TRUE, "description": "contains personal email addresses"} ###text to review [@Value(pdf_page_text)] We are creating a prompt to ask the Local AI to tell us, True or False if the text contains any GDPR information, and if it does, give us a description of what it might be. ### [](#id-4-configure-the-localgenerativeaicaller) 4) Configure the LocalGenerativeAICaller Open the LocalGenerativeAICaller transformer next to the AttributeCreator. Next to 'model:' we want to enter the model we pulled using Ollama earlier, Google Gemma 2. Paste in: Copy gemma2:2b Next, under the Parameter prompt in the transformer, select the Attribute Value 'prompt' you just created using the AttributeCreator. It should look something like this: ### [](#id-5-run-the-workspace) 5) Run the Workspace Run the workspace. Inspect the Visual Preview of the AttributeManager, the attribute ai\_response should tell you if the two PDFs you sampled contained any GDPR related information. See if they differ in their response. **Note about performance** - you will notice fairly quickly the performance is not super quick. We are using a fairly small virtual environment. If you were to run this locally on your own computer, your performance will be better, even if you are using a machine that is a couple of years old. If you are lucky enough to have a dedicated graphics card, this is going to fly! This Workspace uses the Large Language Model (LLM) and runs the model locally through the Avineon Tensing made FME Hub Transformer. [Google Gemma2](https://ollama.com/library/gemma2:9b) [LocalGenerativeIAICaller](https://hub.safe.com/publishers/tensing/transformers/localgenerativeaicaller) [Wikipedia](https://en.wikipedia.org/wiki/General_Data_Protection_Regulation) FME Workspace checking for GDPR sensitive information ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252FZHmjvU9n2JSceqsXtIsU%252Fimage.png%3Falt%3Dmedia%26token%3D895a194a-7e17-4eb0-aef3-3e44a8668801&width=768&dpr=4&quality=100&sign=4737ff56&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252Fb1Wp3PfmLXWQJu1JJ3TH%252Fimage.png%3Falt%3Dmedia%26token%3Dac5e368e-4a49-4ec8-a7da-c3b0c09d4911&width=768&dpr=4&quality=100&sign=eaf41992&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252FO5fp54kBcdmbdxNFIZ0D%252Fimage.png%3Falt%3Dmedia%26token%3D1296dc48-60cc-4e52-b187-e2970196bfe3&width=768&dpr=4&quality=100&sign=31986946&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252Fe4iY7iKLuWZCuNaPZrCz%252Fimage.png%3Falt%3Dmedia%26token%3D0118b1d5-74a0-41d6-9a85-d2592b16aba4&width=768&dpr=4&quality=100&sign=3e5e735a&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252Fy2KKOL2WPoL2Kqj4oxCm%252Fimage.png%3Falt%3Dmedia%26token%3Df687c017-1979-472b-b2a5-11963b6ab693&width=768&dpr=4&quality=100&sign=bb2b7b54&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252F39jiaBxOJXoLbJA4mIer%252Fimage.png%3Falt%3Dmedia%26token%3Df006b977-7a9d-4e3e-b5a5-4a205776bf4c&width=768&dpr=4&quality=100&sign=1695e4ca&sv=2) --- # OpenAI Connectivity | Avineon Tensing UK - FME + Any AI Training Course [PreviousGoogle Gemini](/fme-+-any-ai-training-course/chapter-1-ai-foundations-and-fme-integration/openai-gemini-api-setup/google-gemini) [NextExercise 1.1 - The OpenAIConnector](/fme-+-any-ai-training-course/chapter-1-ai-foundations-and-fme-integration/openai-connectivity/exercise-1.1-the-openaiconnector) Last updated 17 days ago ### [](#integrating-with-openai) Integrating with OpenAI The team at Safe Software and Avineon Tensing have published a number of OpenAI Custom Transformer integrations to help the FME Platform user base get a jump on accessing the OpenAI Platform without having to deal with the complexity of another API. Here's a few of the integrations: ### [](#openaiconnector-transformer-what-does-it-do) OpenAIConnector Transformer - What Does It Do? ### [](#when-would-you-use-it) When Would You Use It? This transformer supports AI-driven tasks such as content generation, classification, summarisation, or transformation of structured/unstructured data, by leveraging OpenAI's text generation capabilities in a repeatable, automatable data workflow. You can: * Extract structured data from unstructured text * Generate summaries of technical reports or documentation * Create natural language descriptions of spatial features * Automate routine writing tasks like metadata generation * Translate content between languages * Classify or categorise text data, such as carrying out sentiment analysis! ### [](#setting-up-the-transformer-and-using-it-effectively) Setting Up the Transformer And Using It Effectively Add the OpenAIConnector to your workspace: * Input your OpenAI **API key** in the API Key parameter. * Select your preferred **Model** (impacts capabilities and cost). * Define your **User Prompt**, bearing in mind that adding system-level instructions that guide the behavior of the model can help, although **Instructions** are optional. * This is the main prompt or query from the user that the model will respond to. This is the core of your interaction and can be a static text string, or dynamically populated using FME attributes with the syntax @Value(AttributeName). Craft this carefully as it directly determines what you'll get back! * Adjust **Temperature** based on how creative vs. precise you want responses (0.00-1.00). * At 0.00: Very focused, consistent and deterministic responses (ideal for factual queries or data extraction). * At 0.50: Balanced creativity and coherence (good default for general use). * At 1.00: More random, creative and varied responses (better for creative writing or brainstorming). **Note:** when making requests to the API, temperature does have a maximum of 2. Proceed with caution when using higher values. Different models may have different amounts of apparent “creativity” and you may need to adjust the temperature accordingly. * Configure **Max Tokens** to control the response length * Sets the maximum length of the response the model will generate. One token is roughly 4 characters or 3/4 of a word. Setting this helps control costs and ensures responses don't become excessively long. For reference, a typical paragraph might be 50-100 tokens Let's give it a go with a little hands-on practice. Whilst these are all currently custom transformers, the history of FME tells us, that there may be a time in the future where these transformers are consolidated into a slightly more official '**package**' based transformer. However, for now they reside in the as downloadable tools. The transformer allows you to send prompts to OpenAI's large language models and receive responses directly within an FME workspace. This model accepts text, image (Vision), file search, web search based and reasoning inputs. Think of it as your direct line of communication to ChatGPT and other OpenAI models, packaged as a convenient FME transformer! Set the **Action** parameter to the type of operation to perform via the . Supported options include: , , , , and . Review the to explore all available models and compare their capabilities. [FME Hub](https://hub.safe.com/) [OpenAIConnector](https://hub.safe.com/publishers/safe-software-verified/transformers/openaiconnector) [Responses API](https://platform.openai.com/docs/api-reference/responses/create) [text generation](https://platform.openai.com/docs/guides/text?api-mode=responses) [image (vision)](https://platform.openai.com/docs/guides/images?api-mode=responses) [file search](https://platform.openai.com/docs/guides/tools-file-search) [web search](https://platform.openai.com/docs/guides/tools-web-search?api-mode=responses) [Reasoning](https://platform.openai.com/docs/guides/reasoning?api-mode=responses) [Model Guide](https://platform.openai.com/docs/models) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252Fe1ImhXbPyoQAJEvaAunI%252Fimage.png%3Falt%3Dmedia%26token%3Df1742672-39fe-40d1-b6fa-88eb86938f0b&width=768&dpr=4&quality=100&sign=ff0f436b&sv=2) --- # Exercise 3.3 - Local AI Translator | Avineon Tensing UK - FME + Any AI Training Course [PreviousExercise 3.2 - GDPR Checker](/fme-+-any-ai-training-course/chapter-3-local-ai/exercise-3.2-gdpr-checker) [NextExercise 3.4 - Sentiment Analysis](/fme-+-any-ai-training-course/chapter-3-local-ai/exercise-3.4-sentiment-analysis) Last updated 16 days ago Local Generative AI, namely with small open source Large Language Models, is a great way to run local translations between different languages. It's also helpful to simplify content for wider audiences. Translating sensitive data whether it be in documents, or in tabular form is possible with FME. You might not be surprised to learn that Safe Software is broadening the appeal of FME at the moment by localising the application interface of FME Form into an increasing number of languages. Of course, the first draft of the interface revision is being done with AI. **Scenario** Translate the schema of a dataset from French to English! **Goals** We're used to using FME as a data translation tool, but a language translation tool is also possible and in this exercise we'll see how that can be done by utilising a Local AI service. **Start Workspace** C:\\TrainingData\\Workspaces\\FMEAnyAI\\Exercise3.3-Complete.fmw **End Workspace** N/A **Data** Electronic components (TXT) ### [](#id-1-open-and-run-the-starting-workspace) **1) Open and Run the Starting Workspace** The top part of the Workspace is taking the column names from the original dataset and, following translation, outputting a text file with some suggested English column names. The lower level flow is translating each record from French to English and outputting to a pipe | separated CSV file. **Note about performance** - you will notice fairly quickly the performance is not super quick. We are using a small virtual environment. If you were to run this locally on your own computer, your performance will be better, even if you are using a machine that is a couple of years old. If you are lucky enough to have a dedicated graphics card, this is going to fly! This example Workspace reads a tabular dataset and translates the columns and records from French into English. Within this example, we use the and transformers which are really powerful for dynamic data pipelines. We'll deep dive even further into the powers of the SchemaScanner later in the course. This Workspace uses the Large Language Model (LLM) to run the model locally through the Avineon Tensing made FME Hub Transformer. [ListExploder](https://docs.safe.com/fme/html/FME-Form-Documentation/FME-Transformers/Transformers/listexploder.htm) [SchemaScanner](https://docs.safe.com/fme/html/FME-Form-Documentation/FME-Transformers/Transformers/schemascanner.htm) [Google Gemma2](https://ollama.com/library/gemma2:9b) [LocalGenerativeIAICaller](https://hub.safe.com/publishers/tensing/transformers/localgenerativeaicaller) Translating tabular data using Local AI tools within FME ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2F14639024-files.gitbook.io%2F%7E%2Ffiles%2Fv0%2Fb%2Fgitbook-x-prod.appspot.com%2Fo%2Fspaces%252F3gaZkHJOAvL5gXLEsKl6%252Fuploads%252FJVxaOazOBV6hThPmkJ2X%252Fimage.png%3Falt%3Dmedia%26token%3Ded473e66-e4bc-4587-aa0e-faf694cec867&width=768&dpr=4&quality=100&sign=ffd7d70a&sv=2) ![](https://tensing-1.gitbook.io/~gitbook/image?url=https%3A%2F%2Fcontent.gitbook.com%2Fcontent%2F3gaZkHJOAvL5gXLEsKl6%2Fblobs%2FyO3DJk8il8PW61XRlC8N%2Fimage.png&width=768&dpr=4&quality=100&sign=978b5d8b&sv=2) ---