{"id":11191,"date":"2023-12-13T12:52:34","date_gmt":"2023-12-13T17:52:34","guid":{"rendered":"https:\/\/www.jitendrazaa.com\/blog\/?p=11191"},"modified":"2023-12-15T07:02:40","modified_gmt":"2023-12-15T12:02:40","slug":"converting-salesforce-data-into-embeddings-with-openai-and-aws-lambda","status":"publish","type":"post","link":"https:\/\/www.jitendrazaa.com\/blog\/salesforce\/converting-salesforce-data-into-embeddings-with-openai-and-aws-lambda\/","title":{"rendered":"Converting Salesforce Data into Embeddings with OpenAI and AWS Lambda"},"content":{"rendered":"\n<p>Last week on Dec 7, I had an opportunity to speak in the Salesforce Alpharetta Developer Group meeting. This is a 3 part blog series, explaining samples &amp; demos I covered in that group meet. To get information about future sessions, you can join <a href=\"https:\/\/trailblazercommunitygroups.com\/salesforce-developer-group-alpharetta-united-states\/\">this group <\/a>and also the <a href=\"https:\/\/trailhead.salesforce.com\/trailblazer-community\/groups\/0F93A000000HUpbSAG?tab=discussion&amp;sort=LAST_MODIFIED_DATE_DESC\">chatter group to get content<\/a>.<\/p>\n\n\n\n<p>Let&#8217;s start with the basics. <\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What is Embedding?<\/h2>\n\n\n\n<p>Imagine you have a big box of crayons with many different colors. Each crayon is like a word or a picture. Now, suppose you want to organize these crayons so that similar colors are close to each other. For example, different shades of blue are grouped together, and so are the shades of red. This makes it easier for you to find and compare colors.<\/p>\n\n\n\n<p>In the world of AI, embeddings do something similar but with words, pictures, or other kinds of information. Instead of crayons, we have lots of data, like words in a book or images. An embedding is a special method that sorts this data into groups of similar things.<\/p>\n\n\n\n<p>For example, in a book, the words &#8220;happy&#8221; and &#8220;joyful&#8221; might be close together in this sorted space because they mean similar things. This helps AI models quickly understand and work with lots of information, like helping you find a book you might like or understanding what you mean when you talk to a computer.<\/p>\n\n\n\n<p>So, embeddings are like a smart way of organizing a huge box of crayons (or data) so that similar things are close together, making it much easier and faster to use them for different tasks!<\/p>\n\n\n\n<p>If we want to work with AI on Salesforce, very first step that needs to happen is to convert your data to embedding. In this case we would convert Salesforce data to embedding from lambda function using OpenAI API. Eventually, we would use some kind of library like langChain that can retrieve similar data to enrich prompts and get more contextual responses which otherwise would not be possible, this process is known as RAG &#8211; <strong>Retrieval-Augmented Generation<\/strong>.<\/p>\n\n\n\n<!--more-->\n\n\n\n<p><strong>Prerequisite<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Salesforce account with API access<\/li>\n\n\n\n<li>AWS account with Lambda and IAM access<\/li>\n\n\n\n<li>OpenAI API key<\/li>\n\n\n\n<li>Basic understanding of Python programming<\/li>\n<\/ul>\n\n\n\n<p>AWS Lambda requires external libraries to be included as layers. You&#8217;ll need to create layers for <strong>OpenAI, numpy<\/strong>, and <strong>pandas<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Package your Python libraries in a ZIP file.<\/li>\n\n\n\n<li>Upload the ZIP file to Lambda Layers.<\/li>\n\n\n\n<li>Attach these layers to your Lambda function.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Complete Source Code &amp; Video<\/h2>\n\n\n\n<p>Below source code, connects with Salesforce, fetches lead record and uses Open AI to convert data into embedding. It also shows how searched parameter can be compared with matched word using weightage <\/p>\n\n\n\n<script src=\"https:\/\/gist.github.com\/JitendraZaa\/f04ed4c5cfca6c97b97cf57f38eeeab1.js\"><\/script>\n\n\n\n<p>Few Observations<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>I did not use SImple_Salesforce python library as it would need to be imported as a layer in lambda<\/li>\n\n\n\n<li>Environment variables can be created by navigating to the Configuration tab in the lambda<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"AWS Lambda integration with Salesforce\" width=\"500\" height=\"281\" src=\"https:\/\/www.youtube.com\/embed\/zMYI2SiTZ6Y?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share\" referrerpolicy=\"strict-origin-when-cross-origin\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<p>Below is the screenshot of the outcome<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" width=\"752\" height=\"554\" src=\"https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2023\/12\/AWS-Lambda-Embedding.png?resize=752%2C554&#038;ssl=1\" alt=\"\" class=\"wp-image-11194\" srcset=\"https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2023\/12\/AWS-Lambda-Embedding.png?w=752&amp;ssl=1 752w, https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2023\/12\/AWS-Lambda-Embedding.png?resize=300%2C221&amp;ssl=1 300w\" sizes=\"auto, (max-width: 752px) 100vw, 752px\" \/><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>Discover how to leverage OpenAI&#8217;s powerful embedding capabilities to transform your Salesforce data into insightful, actionable embeddings. This guide provides a step-by-step process, complete with code, to seamlessly integrate Salesforce data with OpenAI&#8217;s advanced AI models, unlocking new dimensions of data analysis and decision-making.<\/p>\n","protected":false},"author":1,"featured_media":11232,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"jetpack_post_was_ever_published":false,"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"jz_research_post":"","_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[9],"tags":[528,554,556,555,557,527,560,212],"class_list":["post-11191","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-salesforce","tag-ai","tag-aws","tag-embedding","tag-lambda","tag-openai","tag-python","tag-rag","tag-video-tutorial"],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2023\/12\/Salesforce-Langchain-Python-OpenAI-AWS-Lambda1.png?fit=1120%2C630&ssl=1","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":11235,"url":"https:\/\/www.jitendrazaa.com\/blog\/salesforce\/talk-to-salesforce-data-using-openai-langchain-chroma\/","url_meta":{"origin":11191,"position":0},"title":"Talk to Salesforce Data Using OpenAI, Langchain &#038; Chroma","author":"Jitendra","date":"December 14, 2023","format":false,"excerpt":"Discover how to leverage OpenAI's powerful embedding capabilities to transform your Salesforce data into insightful, actionable embeddings. This guide provides a step-by-step process, complete with code, to seamlessly integrate Salesforce data with OpenAI's advanced AI models, unlocking new dimensions of data analysis and decision-making. It uses Langchain and Chroma vector\u2026","rel":"","context":"In &quot;Salesforce&quot;","block_context":{"text":"Salesforce","link":"https:\/\/www.jitendrazaa.com\/blog\/category\/salesforce\/"},"img":{"alt_text":"Salesforce Langchain Python OpenAI AWS Lambda2","src":"https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2023\/12\/Salesforce-Langchain-Python-OpenAI-AWS-Lambda2.png?fit=1120%2C630&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2023\/12\/Salesforce-Langchain-Python-OpenAI-AWS-Lambda2.png?fit=1120%2C630&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2023\/12\/Salesforce-Langchain-Python-OpenAI-AWS-Lambda2.png?fit=1120%2C630&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2023\/12\/Salesforce-Langchain-Python-OpenAI-AWS-Lambda2.png?fit=1120%2C630&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2023\/12\/Salesforce-Langchain-Python-OpenAI-AWS-Lambda2.png?fit=1120%2C630&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":11240,"url":"https:\/\/www.jitendrazaa.com\/blog\/salesforce\/talk-to-salesforce-release-notes-using-open-ai-langchain-chroma\/","url_meta":{"origin":11191,"position":1},"title":"Talk to any PDF document using AI","author":"Jitendra","date":"December 14, 2023","format":false,"excerpt":"Discover how to leverage OpenAI's powerful embedding capabilities to transform your Salesforce data into insightful, actionable embeddings. This guide provides a step-by-step process, complete with code, to seamlessly integrate Salesforce data with OpenAI's advanced AI models, unlocking new dimensions of data analysis and decision-making. We would be using Salesforce Release\u2026","rel":"","context":"In &quot;Salesforce&quot;","block_context":{"text":"Salesforce","link":"https:\/\/www.jitendrazaa.com\/blog\/category\/salesforce\/"},"img":{"alt_text":"Salesforce Langchain Python OpenAI AWS Lambda3","src":"https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2023\/12\/Salesforce-Langchain-Python-OpenAI-AWS-Lambda3.png?fit=1120%2C630&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2023\/12\/Salesforce-Langchain-Python-OpenAI-AWS-Lambda3.png?fit=1120%2C630&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2023\/12\/Salesforce-Langchain-Python-OpenAI-AWS-Lambda3.png?fit=1120%2C630&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2023\/12\/Salesforce-Langchain-Python-OpenAI-AWS-Lambda3.png?fit=1120%2C630&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2023\/12\/Salesforce-Langchain-Python-OpenAI-AWS-Lambda3.png?fit=1120%2C630&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":7770,"url":"https:\/\/www.jitendrazaa.com\/blog\/salesforce\/salesforce-integration-with-chatgpt\/","url_meta":{"origin":11191,"position":2},"title":"Salesforce Integration with ChatGPT","author":"Jitendra","date":"April 22, 2023","format":false,"excerpt":"Enhancing Conversational AI Capabilities and Ensuring Security","rel":"","context":"In &quot;Salesforce&quot;","block_context":{"text":"Salesforce","link":"https:\/\/www.jitendrazaa.com\/blog\/category\/salesforce\/"},"img":{"alt_text":"Salesforce integration with chatGPT","src":"https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2023\/04\/Salesforce-integration-with-chatGPT.png?fit=1200%2C675&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2023\/04\/Salesforce-integration-with-chatGPT.png?fit=1200%2C675&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2023\/04\/Salesforce-integration-with-chatGPT.png?fit=1200%2C675&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2023\/04\/Salesforce-integration-with-chatGPT.png?fit=1200%2C675&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2023\/04\/Salesforce-integration-with-chatGPT.png?fit=1200%2C675&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":5231,"url":"https:\/\/www.jitendrazaa.com\/blog\/salesforce\/salesforce-analytics-cloud-basics-video\/","url_meta":{"origin":11191,"position":3},"title":"Salesforce Analytics Cloud Basics &#8211; Video","author":"Jitendra","date":"January 18, 2016","format":false,"excerpt":"Salesforce Analytics Cloud is new Business Intelligence tool from Salesforce on Wave platform. How Wave Works We can import millions of data into wave from external systems like Salesforce, SAP, BOOMI, CSV, Informatica etc. Imported data in Wave is known as Dataset. Multiple lenses can be created on those Datasets\u2026","rel":"","context":"In &quot;Salesforce&quot;","block_context":{"text":"Salesforce","link":"https:\/\/www.jitendrazaa.com\/blog\/category\/salesforce\/"},"img":{"alt_text":"Salesforce Wave Analytics Cloud","src":"https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2016\/01\/Salesforce-Wave-300x79.png?resize=350%2C200&ssl=1","width":350,"height":200},"classes":[]},{"id":6496,"url":"https:\/\/www.jitendrazaa.com\/blog\/salesforce\/embed-lightning-component-in-flow\/","url_meta":{"origin":11191,"position":4},"title":"Embed Lightning Component in Flow","author":"Jitendra","date":"June 24, 2018","format":false,"excerpt":"How to create a lookup field in Flow by embedding Lightning Component","rel":"","context":"In &quot;Salesforce&quot;","block_context":{"text":"Salesforce","link":"https:\/\/www.jitendrazaa.com\/blog\/category\/salesforce\/"},"img":{"alt_text":"Use Lightning Component in Salesforce Flow","src":"https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2018\/06\/Screen-Shot-2018-06-24-at-10.52.30-PM.png?fit=1200%2C412&ssl=1&resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2018\/06\/Screen-Shot-2018-06-24-at-10.52.30-PM.png?fit=1200%2C412&ssl=1&resize=350%2C200 1x, https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2018\/06\/Screen-Shot-2018-06-24-at-10.52.30-PM.png?fit=1200%2C412&ssl=1&resize=525%2C300 1.5x, https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2018\/06\/Screen-Shot-2018-06-24-at-10.52.30-PM.png?fit=1200%2C412&ssl=1&resize=700%2C400 2x, https:\/\/i0.wp.com\/www.jitendrazaa.com\/blog\/wp-content\/uploads\/2018\/06\/Screen-Shot-2018-06-24-at-10.52.30-PM.png?fit=1200%2C412&ssl=1&resize=1050%2C600 3x"},"classes":[]},{"id":6800,"url":"https:\/\/www.jitendrazaa.com\/blog\/salesforce\/video-large-data-volume-how-lightning-platform-query-optimizer-works\/","url_meta":{"origin":11191,"position":5},"title":"Video &#8211; Large Data Volume &#8211; How Lightning Platform Query Optimizer Works","author":"Jitendra","date":"March 5, 2019","format":false,"excerpt":"Best Practices of handling Large Data Volume (LDV) in Salesforce","rel":"","context":"In &quot;Salesforce&quot;","block_context":{"text":"Salesforce","link":"https:\/\/www.jitendrazaa.com\/blog\/category\/salesforce\/"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/img.youtube.com\/vi\/FtfZV5uf9fg\/0.jpg?resize=350%2C200","width":350,"height":200},"classes":[]}],"jetpack_likes_enabled":true,"_links":{"self":[{"href":"https:\/\/www.jitendrazaa.com\/blog\/wp-json\/wp\/v2\/posts\/11191","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.jitendrazaa.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.jitendrazaa.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.jitendrazaa.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.jitendrazaa.com\/blog\/wp-json\/wp\/v2\/comments?post=11191"}],"version-history":[{"count":4,"href":"https:\/\/www.jitendrazaa.com\/blog\/wp-json\/wp\/v2\/posts\/11191\/revisions"}],"predecessor-version":[{"id":11257,"href":"https:\/\/www.jitendrazaa.com\/blog\/wp-json\/wp\/v2\/posts\/11191\/revisions\/11257"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.jitendrazaa.com\/blog\/wp-json\/wp\/v2\/media\/11232"}],"wp:attachment":[{"href":"https:\/\/www.jitendrazaa.com\/blog\/wp-json\/wp\/v2\/media?parent=11191"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.jitendrazaa.com\/blog\/wp-json\/wp\/v2\/categories?post=11191"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.jitendrazaa.com\/blog\/wp-json\/wp\/v2\/tags?post=11191"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}