0

Explore Products

Best Sellers

New Releases

Books

Videos

Audiobooks

Free Learning

Blog .NET for Apache Spark UDFs Missing Shared State from Blog Posts - SQLServerCentral

Anonymous

1 min read
29 Oct 2020

0 Likes
0 Comments
804 Views

article-image

The Problem When you use a UDF in .NET for Apache Spark, something like this code:

class Program { static void Main(string[] args) { var spark = SparkSession.Builder().GetOrCreate(); _logging.AppendLine("Starting Select"); var udf = Functions.Udf<int, string>(theUdf); spark.Range(100).Select(udf(Functions.Col("id"))).Show(); _logging.AppendLine("Ending Select"); Console.WriteLine(_logging.ToString()); } private static readonly StringBuilder _logging = new StringBuilder(); private static string theUdf(int val) { _logging.AppendLine($"udf passed: {val}"); return $"udf passed {val}"; } } Generally, knowing .NET we would expect the following output:

The post Blog .NET for Apache Spark UDFs Missing Shared State appeared first on SQLServerCentral.

Like
Save for later
Comment

0 Likes
0 Comments
804 Views

Recommendations for you

LLM Engineer's Handbook

LLM Engineer's Handbook

Oct 2024 522 pages

eBook

€43.99

Raspberry Pi and MQTT Essentials

Raspberry Pi and MQTT Essentials

Sep 2022 272 pages

eBook

€19.99 ~~€22.99~~

Full-Stack Flask and React

Full-Stack Flask and React

Oct 2023 408 pages

eBook

€20.99 ~~€23.99~~

Fundamentals of Object-Oriented Programming - C++

Fundamentals of Object-Oriented Programming - C++

Feb 2023 7hrs 6mins

Video

€22.99

Getting Started with Tableau 2018.x

Getting Started with Tableau 2018.x

Sep 2018 396 pages

eBook

€28.99 ~~€32.99~~

Semantic Kernel SDK for Intelligent Applications

Semantic Kernel SDK for Intelligent Applications

Dec 2024 2hrs 9mins

Video

€82.99

AI Ecosystem for the Absolute Beginners - Hands-On

AI Ecosystem for the Absolute Beginners - Hands-On

Dec 2024 4hrs 56mins

Video

€81.99

CompTIA Security+ SY0-701 Certification Guide

CompTIA Security+ SY0-701 Certification Guide

Jan 2024 634 pages

Paperback

€33.99

Python for Algorithmic Trading Cookbook

Python for Algorithmic Trading Cookbook

Aug 2024 404 pages

eBook

€31.99 ~~€35.99~~

C# 13 and .NET 9 – Modern Cross-Platform Development Fundamentals

C# 13 and .NET 9 – Modern Cross-Platform Development Fundamentals

Nov 2024 828 pages

eBook

€26.98 ~~€29.99~~

article-image-revolutionising-work-and-everyday-life-with-chatgpt

M.T. White

16 Dec 2024

Revolutionising Work and Everyday Life with ChatGPT

M.T. White

16 Dec 2024

10 min read

article-image-building-trust-in-ai-the-role-of-rag-in-data-security-and-transparency

Keith Bourne

13 Dec 2024

Building Trust in AI: The Role of RAG in Data Security and Transparency

Keith Bourne

13 Dec 2024

15 min read

article-image-enhancing-data-quality-with-cleanlab

Prakhar Mishra

11 Dec 2024

Enhancing Data Quality with Cleanlab

Prakhar Mishra

11 Dec 2024

10 min read

article-image-revolutionize-power-bi-queries-with-openai

Gus Frazer

11 Dec 2024

Revolutionize Power BI Queries with OpenAI

Gus Frazer

11 Dec 2024

10 min read

Comments (0)

No comments for this article yet!