Tech News - Languages

Yesterday, the Rust team shared that a Cargo vulnerability confuses the older versions of Cargo making them ignore the new package rename feature and download a wrong dependency. This vulnerability, tracked as CVE-2019-16760, affects Rust 1.0 through Rust 1.25. The vulnerability was first reported to the Rust team by Elichai Turkel: https://twitter.com/Elichai2/status/1178681807170101248 Details of Cargo vulnerability Rust 1.31 introduced the package configuration key for renaming dependencies in the ‘Cargo.toml’ manifest file. In Rust 1.25 and prior, Cargo ignores its usage to rename dependencies and may end up downloading a wrong dependency. It affects not only manifests that are written locally, but also those that are published to crates.io. “If you published a crate, for example, that depends on `serde1` to crates.io then users who depend on you may also be vulnerable if they use Rust 1.25.0 and prior. Rust 1.0.0 through Rust 1.25.0 is affected by this advisory because Cargo will ignore the `package` key in manifests,” the team wrote. This vulnerability does not affect Rust 1.26 through Rust 1.30 versions and will throw an error as the package key is unstable in these versions. Rust 1.31 and later are not affected because Cargo understands the package key. Mitigation steps to prevent this Cargo vulnerability The team has already audited the existing crates using the package key published to crates.io and have not detected any exploit of this vulnerability. However, they have recommended users of the affected versions to update their compiler to either 1.26 or later. The team further wrote, “We will not be issuing a patch release for Rust versions prior to 1.26.0. Users of Rust 1.19.0 to Rust 1.25.0 can instead apply the provided patches to mitigate the issue.” This news sparked a discussion on Reddit where developers discussed how this could have been avoided. A user commented, “What do we learn from this? Always throw an error if you encounter an unknown key inside a known configuration object.” Another user suggested, “It would be better to have the config contain a "minimum allowed cargo version", and if you want to use new features you have to bump this version number to at least the version which added the feature. Old versions of cargo can detect the version number and automatically refuse to compile the crate if the minimum version is newer than the cargo version.” Read the official announcement by the Rust team to know more about this vulnerability in detail. Rust 1.38 releases with pipelined compilation for better parallelism while building a multi-crate project Mozilla introduces Neqo, Rust implementation for QUIC, new http protocol Introducing Weld, a runtime written in Rust and LLVM for cross-library optimizations  

After releasing Rust 1.37 last month, the Rust team announced the release of Rust 1.38 yesterday. This version supports pipelined rustc compilation, an extended #[deprecated] attribute for macros, the std::any::type_name function to know the type name, and more. Key updates in Rust 1.38 Pipelined compilation to increase parallelism Rust’s dependency manager and project compiler, Cargo create a directed acyclic graph  (DAG) of crates whenever a cargo build command is fired. Cargo waits till all of the dependencies for that compilation is completed, only then it proceeds to execute rustc.  Starting with 1.38, this wait is minimized by introducing pipelined compilation. Cargo will now take advantage of the metadata produced by the compiler to start the next compilation. Read also: Rust 1.29 is out with improvements to its package manager, Cargo The team shared in the announcement that though this update doesn’t have much effect on builds for a single crate, it has shown up to 10-20% improvement in compilation speed during testing. “Other ones did not improve much, and the speedup depends on the hardware running the build, so your mileage might vary. No code changes are needed to benefit from this,” the team adds. Linting incorrect uses of mem::{uninitialized, zeroed} Previously, the ‘mem::uninitialized’ function allowed developers to sidestep Rust’s initialization checks. This operation can be “incredibly dangerous” as it makes the Rust compiler assume that values are properly initialized. This was addressed in Rust 1.36 by stabilizing the ‘MaybeUninit<T>’ type. The Rust team explained in a previous announcement, “The Rust compiler will understand that it should not assume that a MaybeUninit<T> is a properly initialized T. Therefore, you can do gradual initialization more safely and eventually use .assume_init() once you are certain that maybe_t: MaybeUninit<T> contains an initialized T.” The ‘mem::uninitialized’ function is planned to deprecate in Rust 1.39. Starting with Rust 1.38, the compiler has a few checks to identify incorrect initializations using ‘mem::uninitialized’ or ‘mem::zeroed’. However, these checks do not cover all cases of unsound use of these methods. The #[deprecated] attribute for macros Rust 1.9 introduced the  #[deprecated] attribute that allows crate authors to notify their users an item of their crate is deprecated and will be removed in a future release. In Rust 1.38, this attribute can also be applied to indicate the deprecation of macros. The std::any::type_name function Rust 1.38 introduces a new function called ‘std::any::type_name’ that gives you the type name. Since this is a standard library function for debugging, the exact content and format of the string are not guaranteed. The team explains, “The value returned is only a best-effort description of the type; multiple types may share the same type_name value, and the value may change in future compiler releases.” Some users have already upgraded to Rust 1.38 and shared their feedback on Twitter. https://twitter.com/gilescope/status/1177229413320134661 While others are eagerly waiting for the release that will have the stabilized async-await. https://twitter.com/shirshak55/status/1177247635285000192 These were some of the updates in Rust 1.38. Check out the official announcement for more. Introducing Weld, a runtime written in Rust and LLVM for cross-library optimizations Mozilla introduces Neqo, Rust implementation for QUIC, new http protocol Introducing Nushell: A Rust-based shell Oracle releases JDK 13 with switch expressions and text blocks preview features, and more! Darklang available in private beta

Yesterday the Mypy team uploaded mypy 0.730 to the Python Package Index (PyPI). Mypy is a static type checker for Python. This release includes many features, bug fixes and library stub updates. You can install it as follows: python3 -m pip install -U mypy Python creator Guido van Rossum tweeted about this release, he says “mypy 0.720 released. New semantic analyzer is now the default! Also, --warn-unreachable flag, and many more fixes and updates.” Major updates in Mypy 0.730 Some of the breaking changes in the Mypy 0.730 include: More precise error locations If you call a function with an incompatible argument type, mypy now points the error message to the argument with the incompatible type. Previously, mypy pointed to the function being called, which could be confusing in multi-line calls. Error codes Mypy 0.730 can now optionally display error codes. They are shown within square brackets after each error message: prog.py:24: error: "str" has no attribute "trim"  [attr-defined] Enable error codes using --show-error-codes (or show_error_codes = True in a configuration file). Ignoring specific error codes You can ignore only errors with specific error codes on a particular line by using a # type: ignore[code, ...] comment. This reduces the risk of ignoring unexpected, serious errors when using # type: ignore comments that ignore (almost) all possible errors on a line. Colors in output Mypy 0.730 now uses colored, more user-friendly output by default. You can use --no-color to disable colored output. You can use --no-error-summary to hide the summary line with the number of errors. Pretty output mode You can use --pretty to display each line which had errors and a caret that points to the location of the error on each line. Old semantic analyzer removed This release of Mypy 0.730 no longer includes the old semantic analyzer. Reachability and context managers The --warn-unreachable option now behaves more correctly with “exception-swallowing” context managers. If a context manager is currently declared to return bool but it never swallows exceptions, you should annotate the return of __exit__ as Literal[False] instead of bool, or otherwise mypy may complain about missing return statements. To know more about this release, read the full documentation on Read the Docs. Other interesting news in programming Łukasz Langa at PyLondinium19: “If Python stays synonymous with CPython for too long, we’ll be in big trouble” Microsoft introduces Pyright, a static type checker for the Python language written in TypeScript Python 3.8 alpha 2 is now available for testing

Yesterday, the team at Nim announced Nim version 1.0, a general-purpose, compiled programming language that focuses on efficiency, readability and flexibility. Major changes in Nim 1.0 Backwards compatibility The switch -d:nimBinaryStdFiles has been removed in this release and stdin/stdout/stderr are now the binary files again.  In this release, the language definition and compiler are now stricter about gensym'ed symbols in hygienic templates.  Changes made to library The team has removed unicode.Rune16 in this release as the name ‘Rune16 ’ was wrong. In Nim 1.0, encodings.getCurrentEncoding distinguishes between the OS's encoding and console's encoding.  In this release, json.parseJsonFragments iterator can speedup JSON processing. Oid usage has been enabled in hashtables. std/monotimes module has been added that implements monotonic timestamps. Compiler In Nim 1.0, the Nim compiler warns about unused module imports. Users can use a top-level {.used.} pragma in the module that can be importable without giving a warning. In this version, the Nim compiler nomore recompiles the Nim project via nim c -r if case no dependent Nim file is changed. Users seem to be excited about this news and are appreciating the efforts taken by the team. A user commented on HackerNews, “Great! I love this language, so simple and powerful, so fast executables!”  Another user commented, “I would have never thought to live long enough to see this happening! I started using Nim in 2014, but abandoned it after a few years, frustrated by the instability of the language and what I perceived as a lack of vision. (In 2014, release 1.0 was said to be "behind the corner".) This release makes me eager to try it again. I remember that the language impressed me a lot: easy to learn, well-thought, and very fast to compile. Congratulations to the team!” Other interesting news in programming How Quarkus brings Java into the modern world of enterprise tech LLVM 9 releases with official RISC-V target support, asm goto, Clang 9, and more Twitter announces to test ‘Hide Replies’ feature in the US and Japan, after testing it in Canada

Last week, Michael Thomsen, the Project Manager for Dart announced the stable release of Dart 2.5 SDK (Software Development Kit). This release includes two technical previews of ML Complete and the dart:ffi foreign function interface to be used for calling C code directly from Dart. Dart 2.5 also brings improved support for constant expressions. Preview of ML Complete In his blog, Thomsen has regarded ML Complete as a “powerful addition” to their existing suite of productivity tools like hot reload, customizable static analysis, and Dart DevTools. It works by training a model of possible member occurrences in a given context. The possible occurrences can be analyzed from the available open-source Dart code on Github. The training model uses TensorFlow Lite tools to predict the next probable symbol, while the developer is editing. As ML Complete is built directly into the Dart analyzer, it is available on all Dart-enabled editors including Android Studio, IntelliJ, and VS Code. Since it is still in preview, developers are advised to use the Flutter dev channel or the Dart dev channel for previewing this feature. Preview of the dart:ffi foreign function interface The dart:ffi feature enables users to take advantage of the existing native APIs, where Dart code is already running. Users can also utilize the existing cross-platform native libraries written in C. Currently, the support for calling C directly from Dart is limited to the Dart VM deep integration which uses native extensions. The new dart:ffi foreign function interface will function on a new mechanism, offering great performance, easy approach, and will work across many Dart supported platforms and compilers. Dart-C interop works on two main scenarios: Calling a C-based system API on the host operating system (OS) For calling a C-based system API, the Linux command system is used. The system command allows the execution of any system command. It also allows the argument to be essentially passed to the shell/terminal and also run there. For implementing the dart:ffi, the Dart code needs to represent the C function and the types of its arguments and return type. It also needs to represent the corresponding Dart function, and its types. Both the representations are done by defining two typedefs in the C header for the command. Calling a C-based library for a single OS or cross-platform The dart:ffi feature is also used to implore C-based frameworks and components. It will allow the user to run TensorFlow across all the operating systems where code completion is needed. It also offers high performance of the native TensorFlow implementation. Thomsen adds, “We also expect that the ability to call C-based libraries will be of great use to Flutter apps. You can imagine calling native libraries such as Realm or SQLite, and we think dart:ffi will be valuable for enabling plugins for Flutter desktop.” Developers are advised to use the Flutter master channel or a Dart dev channel to quickly learn about the changes and improvements in the dart:ffi feature. Read Also: Dart 2.2 is out with support for set literals and more! Improvements in constant expressions In the earlier versions, the abilities of constant expressions were limited, however, Dart 2.5 includes many new changes. In Dart 2.5, constant expressions can be defined using many ways, which includes the ability to use casts and the new control flow and collection spread features. Image Source: Medium Users love Dart 2.5 features. https://twitter.com/geek_timofey/status/1171507571372380167 https://twitter.com/Fredrikkerlund2/status/1171461649217097728 https://twitter.com/RashidG92908642/status/1171910418807369728 To know more about this announcement in detail, visit Michael Thomsen’s blog on Medium. Other Interesting News in Programming Microsoft releases Cascadia Code version 1909.16, the latest monospaced font for Windows Terminal and Visual Studio Code Microsoft open-sources its C++ Standard Library (STL) used by MSVC tool-chain and Visual Studio Linux 5.3 releases with support for AMD Navi GPUs, Zhaoxin x86 CPUs and power usage improvements

Last week, Microsoft open-sourced its implementation of the C++ Standard Library, also known as STL. The library is shipped along with MSVC (Microsoft Visual C++ compiler) toolset and the Visual Studio IDE. This announcement was made by MSVC team at the CppCon 2019 conference, two days ago. Users can use the C++ library repo for participating in the STL's development by reporting issues and commenting on pull requests. The MSVC team is still working on migrating the C++ Standard Library to GitHub. Currently, the Github repository contains all of MSVC's product source code including a new CMake build system and a README. The team also plans to use the GitHub issues to track C++20 features, LWG issues, conformance bugs, performance improvements, and other todos. The roadmap and iteration plans of the C++ Standard Library is also under progress. Why Microsoft open-sourced the C++ Standard Library? Microsoft has open-sourced STL to allow it’s users easy access to all the latest developments in C++ by trying out latest changes and improving pull requests by reviewing them. The MSVC team hopes that as C++ standardization accelerates, it will be easier for users to accept the major features. Microsoft chose to open-source STL particularly due to its unique design and fast-evolving nature when compared to other MSVC libraries and compiler. It is also “easy to contribute to, and somewhat loosely coupled, unlike the compiler.” The official blog post adds, “We also want to contribute back to the C++ community by making it possible to take our implementations of major features.” What are the primary goals of the C++ Standard Library? Microsoft is implementing the latest C++ Working Draft, which will eventually become the next C++ International Standard. The goals of the Microsoft C++ Standard Library are to be conformant to spec, extremely fast, usable, and extensive compatibility. Speed being the core strength of C++, STL needs to be extremely fast at runtime. Thus, the MSVC team spends more time on the optimization of the C++ Standard Library than the most general-purpose libraries. They are also working on parts of the programming experience like compiler throughput, diagnostic messages, and debugging checks. They are also keeping VS 2019 binary-compatible with VS 2017 and VS 2015. They consider source compatibility to be important, but not all-important; breaking source compatibility can be an acceptable cost if done for the right reasons in the right way. The blog post states that MSVC’s STL is distributed under the Apache License v2.0 with LLVM Exceptions and is distinct from the libc++ library. However, if any libc++’s maintainers are interested in taking feature implementations from MSVC’s STL or in collaborating on the development of new features in both libraries simultaneously, the MSVC team will help irrespective of the licensing. Users have welcomed Microsoft’s move to open-source it’s C++ Standard Library (STL). A Redditor says, “Thank you! Absolutely amazing. It's been one of my guilty pleasures ever since I started with C++ to prod about in your internals to see how stuff works so this is like being taken to the magical chocolate factory for me.” Another user comments, “thank you for giving back to the open source world. ❤🤘” Interested readers can learn how to build with the Native Tools Command Prompt and a Visual Studio IDE on Github. Latest news in Tech Open AI researchers advance multi-agent competition by training AI agents in a simple hide and seek environment As Kickstarter reels in the aftermath of its alleged union-busting move, is the tech industry at a tipping point? Linux 5.3 releases with support for AMD Navi GPUs, Zhaoxin x86 CPUs and power usage improvements

The stable release of LLVM 9.0 is expected to come in the next few weeks along with subprojects like Clang 9.0. As per the release notes, the upcoming Clang 9.0 release will come with experimental support for C++17 features in OpenCL, asm goto support, and much more. Read also: LLVM 9.0 RC3 is now out with official RISC-V support, updates to SystemZ and more What’s new coming in Clang 9.0.0 Experimental support for C++17 features in OpenCL Clang 9.0.0 will have experimental support for C++17 features in OpenCL. The experimental support includes improved address space behavior in the majority of C++ features. There is support for OpenCL-specific types such as images, samplers, events, and pipes. Also, the invoking of global constructors from the host side is possible using a specific, compiler-generated kernel. C language updates in Clang Clang 9.0.0 includes the __FILE_NAME__ macro as a Clang specific extension that is supported in all C-family languages. It is very similar to the __FILE__ macro except that it will always provide the last path component when possible. Another C language-specific update is the initial support for asm goto statements to control flow from inline assembly to labels. This construct will be mainly used by the Linux kernel (CONFIG_JUMP_LABEL=y) and glib. Building Linux kernels with Clang 9.0 With the addition of asm goto support, the mainline Linux kernel for x86_64 is now buildable and bootable with Clang 9. The team adds, “The Android and ChromeOS Linux distributions have moved to building their Linux kernels with Clang, and Google is currently testing Clang built kernels for their production Linux kernels.” Read also: Linux 4.19 kernel releases with open arms and AIO-based polling interface; Linus back to managing the Linux kernel Build system changes Previously, the install-clang-headers target used to install clang’s resource directory headers. With Clang 9.0, this installation will be done by the install-clang-resource-headers target. “Users of the old install-clang-headers target should switch to the new install-clang-resource-headers target. The install-clang-headers target now installs clang’s API headers (corresponding to its libraries), which is consistent with the install-llvm-headers target,” the release notes read. To know what else is coming in Clang 9.0, check out its official release notes. Other news in Programming Core Python team confirms sunsetting Python 2 on January 1, 2020 Developers from the Swift for TensorFlow project propose adding first-class differentiable programming to Swift Microsoft introduces Static TypeScript, as an alternative to embedded interpreters, for programming MCU-based devices

Yesterday, Oracle announced the general availability of Java SE 13 (JDK 13) and that its binaries are expected to be available for download today. In addition to improved performance, stability, and security, this release comes with two preview features, switch expressions and text blocks. This announcement coincides with the commencement of Oracle’s co-located OpenWorld and Code One conferences happening from September 16-17 2019 at San Francisco. Oracle’s director of Java SE Product Management, Sharat Chander, wrote in the announcement, “Oracle offers Java 13 for enterprises and developers. JDK 13 will receive a minimum of two updates, per the Oracle CPU schedule, before being followed by Oracle JDK 14, which is due out in March 2020, with early access builds already available.” This release is licensed under the GNU General Public License v2 with the Classpath Exception (GPLv2+CPE). For those who are using Oracle JDK release as part of an Oracle product or service, it is available under a commercial license. Read also: Oracle releases open-source and commercial licenses for Java 11 and later What’s new in JDK 13 JDK 13 includes the implementation of the following Java Enhancement Proposals (JEPs): Dynamic class-data sharing archives (JEP 350) JEP 350 improves the usability of application class-data sharing to allow the dynamic archiving of classes once the execution of a Java application is completed. The archived classes will consist of all loaded application classes and library classes that are not present in the default, base-layer CDS archive. Uncommit unused memory (JEP 351) Previously, the z garbage collector did not uncommit and returned memory to the operating system, even if it was left unused for a long time. With JEP 351 implemented in JDK 13, the z garbage collector will return unused heap memory to the operating system. Read also: Getting started with Z Garbage Collector (ZGC) in Java 11 [Tutorial] Reimplement the Legacy Socket API (JEP 353) In JDK 13, the underlying implementation used by the ‘java.net.Socket’ and ‘java.net.ServerSocket APIs’ is replaced by “a simpler and more modern implementation that is easy to maintain and debug,” as per JEP 353. This new implementation aims to make adapting to user-mode threads or fibers, that is currently being explored in Project Loom, much easier. Switch expressions preview (JEP 354) The switch expressions feature proposed in JEP 354 allows using ‘switch’ as both a statement or an expression. Developers will now be able to use both the traditional ‘case ... : labels’ (with fall through) or new ‘case ... -> labels’ (with no fall through). This preview feature in JDK 13 aims to simplify everyday coding and prepare the way for the use of pattern matching (JEP 305) in a switch. Text blocks preview (JEP 355) The text blocks preview feature proposed in JEP 355 makes it easy to express strings that take up several source code lines. This preview feature aims to improve both “the readability and the writeability of a broad class of Java programs to have a linguistic mechanism for denoting strings more literally than a string literal.” Check out the official announcement by Oracle to know what else has landed in JDK 13. Other news in programming Microsoft introduces Static TypeScript, as an alternative to embedded interpreters, for programming MCU-based devices Golang 1.13 module mirror, index, and Checksum database are now production-ready Why Perl 6 is considering a name change?

Yesterday, the team behind Dark programming language has unveiled Darklang’s private beta version. Dark is a holistic programming language, editor, and infrastructure for building backends. Developers can write in the Dark language, using the Dark editor, and the program is hosted on Dark’s infrastructure. As a result, they can code without thinking about infrastructure, and have safe instant deployment, which the team is calling “deployless” development. According to the team, backends today are too complicated to build and they have designed Dark in a way to reduce that complexity. Ellen Chisa, CEO of the Dark says, “Today we’re releasing two videos showing how Dark works. And demonstrate how to build a backend application (an office sign-in app) in 10 minutes.” Paul Biggar, the CTO also talks about the Dark’s philosophy and the details of the language, the editor and the infrastructure. He also shows how they make “deployless” safe with feature flags and versioning, and how Dark allows to introspect and debug live requests. Alpha users of Darklang build backends for web and mobile applications The Dark team says that during the private alpha, developers have built entire backends in Dark. Chase Olivieri built Altitude, a flight deal subscription site. Julius Tarng moved the backend of Tokimeki Unfollow to Dark for scalability. Jessica Greenwalt & Pixelkeet ported Birb, their internal project tracker, into a SaaS for other design studios to use. The team has also seem alpha users build backends for web and mobile applications, internal tools, Slackbots, Alexa skills, and personal projects. And they’ve even started building parts of Dark in Dark, including their presence service and large parts of the signup flow. Additionally, the team will let you in the private beta of Darklang immediately if the developers have their project well-scoped and ready to get started. Community unhappy with private version, and expect open-source On Hacker News, users are discussing that in this time and age if there is any new programming language, it has to be open-source. One of them commented, “Is there an open source version of the language? ...bc I'm not touching a programming language with a ten foot pole if it hasn't got at least two implementations, and at least one open source :| Sure, keep the IDEs and deployless infrastructure and all proprietary, but a core programming language in 2019 can only be open-source. Heck, even Microsoft gets it now.” Another one says, “They are 'allowing' people into a private beta of a programming language? Coupled with the fact it is not open source and has a bunch of fad ad-tech videos on the front page this is so many red flags.” While others compare Dark with different programming languages, mainly Apex, Rust and Go. A user comment reads, “I see a lot of Parse comparisons, but for me this is way more like Force.com from Salesforce and the Apex language. Proprietary language (Apex, which is Java 6-ish), complete vertical integration, no open source spec or implementation.” Another one says, “Go - OK, it has one implementation (open-source), but it's backed by one big player (Google) and used by many others... also the simplicity at core design decisions sound like the kind of choices that would make an alternative compiler easier to implement than for other languages Rust - pretty fast growing open-source community despite only one implementation... but yeah I'm sort of worried that Rust is a "hard to implement" kind of language with maybe a not high enough bus factor... similar worries for Julia too But tbh I'm not drawn much to either Go and Rust for other reasons - Go is too verbose for my taste, no way to write denser code that highlights the logic instead of the plumbing, and it has a "dumb" type system, Rust seems a really bad choice for rapid prototyping and iteration which is what I care about now.” Other interesting news in programming this week Introducing ‘ixy’, a simple user-space network driver written in high-level languages like Rust, Go, and C#, among others TextMate 2.0, the text editor for macOS releases GNU community announces ‘Parallel GCC’ for parallelism in real-world compilers

Researchers Paul Emmerich et al have developed a new simple user space network driver called ixy. According to the researchers, ixy is an educational user space network driver for the Intel ixgbe family of 10 Gbit/s NICs. Its goal is to show that writing a super-fast network driver can be surprisingly simple in high-level languages like Rust, Go, Java and C# among others. Ixy has no dependencies, high speed, and a simple-to-use interface for applications to be built on it. The researchers have published their findings in a paper titled The Case for Writing Network Drivers in High-Level Programming Languages. Initially, the researchers implemented ixy in C and then successfully implemented the same driver in other high-level languages such as Rust, Go, C#, Java, OCaml, Haskell, Swift, Javascript, and Python. The researchers have found that the Rust driver executes 63% more instructions per packet but is only 4% slower than a reference C implementation. Go’s garbage collector keeps latencies below 100 µs even under heavy load. Network drivers written in C are vulnerable to security issues Drivers written in C are usually implemented in production-grade server, desktop, and mobile operating systems. Though C has features required for low-level systems programming and fine-grained control over the hardware, they have vulnerabilities for security as “they are exposed to the external world or serve as a barrier isolating untrusted virtual machines”. The paper states that the C code “accounts for 66% of the code in Linux, but 39 out of 40 security bugs related to memory safety found in Linux in 2017 are located in drivers. These bugs could have been prevented by using high-level languages for drivers.” Implementing Rust, Go and other high level languages in ixy network driver Rust: A lightweight Rust struct is allocated for each packet that contains metadata and owns the raw memory. The compiler enforces that the object has a single owner and only the owner can access the object. This prevents use-after-free bugs despite using a completely custom allocator. Rust is the only language evaluated in the case study that protects against use-after-free bugs and data races in memory buffers. Go: It has an external memory that is wrapped in slices to provide bounds checks. The atomic package in Go also indirectly provides memory barriers and volatile semantics thus offering stronger guarantees. C#: The researchers have implemented two external memories out of the many available. It offers a more direct way to work with raw memory by offering full support for pointers with no bounds checks and volatile memory access semantics. Java: The researchers have targeted OpenJDK 12 which offers a non-standard way to handle external memory via the sun.misc.Unsafe object that provides functions to read and write memory with volatile access semantics. OCaml: OCaml Bigarrays backed by external memory is used for DMA buffers and PCIe resources, the allocation is done via C helper functions. The Cstruct library from the OCaml allowed researchers to access data in the arrays in a structured way by parsing definitions similar to C struct definitions and generating code for the necessary accessor functions. Haskell: It is a compiled functional language with garbage collection. The necessary low-level memory access functions are available via the Foreign package. Memory allocation and mapping is available via System.Posix.Memory. Swift: Its memory is managed via automatic reference counting, i.e., the runtime keeps a reference count for each object and frees the object once it is no longer in use. It also offers all the features necessary to implement drivers. JavaScript: ArrayBuffers is used to wrap external memory in a safe way, these arrays can then be accessed as different integer types using TypedArrays, circumventing JavaScript’s restriction to floating-point numbers. Memory allocation and physical address translation is handled via a Node.js module in C. Python: For this driver, the implementation was not explicitly optimized for performance and meant as a simple prototyping environment for PCIe drivers and as an educational tool. The researchers have provided primitives for PCIe driver development in Python. Rust is found to be the prime candidate for safer network drivers After implementing the network driver ixy in all high-level languages, the researchers conclude that Rust is the prime candidate for safer drivers. The paper states, “Rust’s ownership based memory management provides more safety features than languages based on garbage collection here and it does so without affecting latency.” Other languages like Go and C# are also a suitable language if the system can cope with sub-millisecond latency spikes due to garbage collection. Other languages like Haskell and OCaml will also be more useful if their performance is less critical than having a safe and correct system. Though Rust performs better than C, it is 4% slower than the C driver. The reason behind is that Rust applies bounds checks while C does not. Another reason is that C does not require a wrapper object for DMA buffers. Image Source: Research paper Users have found the result of this high-level language implementation of network drivers quite interesting. https://twitter.com/matthewwarren/status/1172094036297048068 A Redditor comments, “Wow, Rust and Go performed quite well. Maybe writing drivers in them isn't that crazy” Many developers are also surprised to see the results of this case study, especially the performance of Go and Swift. A comment on Hacker News says, “The fact that Go is slower than C# really amazes me! Not long ago I switched from C# to Go on a project for performance reasons, but maybe I need to go back.” Another Redditor says, “Surprise me a bit that Swift implementation is well below expected. Being Swift a compiled native ARC language, I consider the code must be revised.” Interested readers can watch a video presentation by Paul Emmerich on ‘How to write PCIe drivers in Rust, go, C#, Swift, Haskell, and OCaml’. Also, you can find more implementation details in the research paper. Other News in Tech New memory usage optimizations implemented in V8 Lite can also benefit V8 Google releases Flutter 1.9 at GDD (Google Developer Days) conference Intel’s DDIO and RDMA enabled microprocessors vulnerable to new NetCAT attack

Yesterday, the Eclipse Foundation announced the release of the Jakarta EE 8 full platform, web profile specifications, and related Technology Compatibility Kits (TCKs). This marks the completion of Java EE’s transition to an open and vendor-neutral evolution process. Explaining the vision behind this release, Mike Milinkovich, executive director of the Eclipse Foundation said, “There are tens of thousands of companies with strategic investments in Java EE and over 10 million Java developers globally. The finalization of the Jakarta EE 8 specifications means that the transition of Java EE to our new open, vendor-neutral, and community-based process has been completed, and paves the way for an entirely new era in Java innovation for enterprise and cloud workloads.” Back in 1999, Sun Microsystems developed Java EE under the name Java 2 Enterprise Edition (J2EE), which was rebranded as Java Platform, Enterprise Edition (Java EE) in 2006. When in 2010 Oracle acquired Sun Microsystems, Java EE’s governance and oversight also moved to Oracle. The development of Java EE’s technical specifications was managed under the Java Community Process (JCP), which was tightly vendor-led effort. In order to make Java EE more open, Oracle made the Eclipse Foundation the new steward of enterprise Java. Read also: Eclipse foundation releases updates on its Jakarta EE Rights to Java trademarks Updates in Jakarta EE 8 Jakarta EE 8 has shipped with the same set of technical specifications as Java EE 8, which means developers are not required to make any changes to their Java EE 8 applications or their use of existing APIs. In this release, the team has focused on updating the process used to determine new specs for Jakarta EE that will replace JCP. This new process is called the Jakarta EE Specification Process (JESP), which will be used by the Jakarta EE Working Group for further development of Jakarta EE. It is based on the Eclipse Foundation Specification Process (EFSP) with a few changes. Rhuan Rocha, a Java EE developer, wrote in the announcement, “The goals of JESP is being a process as lightweight as possible, with a design closer to open source development and with code-first development in mind. With this, this process promotes a new culture that focuses on experimentation to evolve these specification based on experiences gained with experimentation.” A key change in this process is that there is no Spec Lead now who had special intellectual property rights under the JCP. In an interview with JAXenter, Milinkovich explained how this process differs from JCP, “The Jakarta EE Specification Process is a level playing field in which all parties are equal, and collaboration is a must. Some of the other significant differences include a code-first approach, rather than a focus on specifications as the starting point. You can also expect a more fully open, collaborative approach to generating specifications, with every decision made collectively by the community.” Along with the release of Jakarta EE 8 specifications, the Eclipse Foundation also announced the certification of Eclipse GlassFish 5.1 as an open-source compatible implementation of the Jakarta EE 8 Platform. To know more in detail, check out the official announcement by the Eclipse Foundation. Other news in programming Is Scala 3.0 a new language altogether? Martin Odersky, its designer, says “yes and no” Core Python team confirms sunsetting Python 2 on January 1, 2020 Go 1.13 releases with error wrapping, TLS 1.3 enabled by default, improved number literals, and more

Yesterday, the team behind Python posted details about the sunsetting of Python 2. As announced before, post January 1, 2020, Python 2 will not be maintained by the Python team. This means that it will no longer receive new features and it will not be improved even if a security problem is found in it. https://twitter.com/gvanrossum/status/1170949978036084736 Why is Python 2 retiring? In the detailed post, the Python team explains that the huge alterations needed in Python 2 led to the birth of Python 3 in 2006. To keep users happy, the Python team kept improving and publishing both the versions together. However, due to some changes that Python 2 couldn’t  handle and scarcity of time required to improve Python 3 faster, the Python team has decided to sunset the second version. The team says, “So, in 2008, we announced that we would sunset Python 2 in 2015, and asked people to upgrade before then. Some did, but many did not. So, in 2014, we extended that sunset till 2020.” The Python team has clearly stated that January 1, 2020 onwards, they will not upgrade or improve the second version of Python even if a fatal security problem crops up in it. Their advice to Python 2 users is to switch to Python 3 using the official porting guide as the former will not support many tools in the future. On the other hand, Python 3 supports graph for all the 360 most popular Python packages. Users can also check out the ‘Can I Use Python 3?’ to find out which tools need to upgrade to Python 3. Python 3 adoption has begun As the end date of Python has been decided earlier on, many implementations of Python have already dropped support for Python 2 or are supporting both Python 2 and 3 for now. Two months ago, NumPy, the library for Python programming language officially dropped support for Python 2.7 in its latest version NumPy 1.17.0. It will only support Python versions 3.5 – 3.7. Earlier this year, pandas 0.24 stopped support for Python 2. Pandas maintainer, Jeff Reback had said, “It's 2019 and Python 2 is slowly trickling out of the PyData stack.” However, not all projects are yet fully on board. There has also been efforts taken to keep Python 2 alive. In August this year, PyPy announced that that they do not plan to deprecate Python 2.7 support as long as PyPy exists. https://twitter.com/pypyproject/status/1160209907079176192 Many users are happy to say goodbye to the second version of Python in favor of building towards a long term vision. https://twitter.com/mkennedy/status/1171132063220502528 https://twitter.com/MeskinDaniel/status/1171244860386480129 A user on Hacker News comments, “In 2015, there was no way I could have moved to Python 3. There were too many libraries I depended on that hadn't ported yet. In 2019, I feel pretty confident about using Python 3, having used it exclusively for about 18 months now. For my personal use case at least, this timeline worked out well for me. Hopefully it works out for most everyone. I can't imagine they made this decision without at least some data backing it up.” Head over to the Python website for more details about about this news. Latest news in Python Introducing OpenDrop, an open-source implementation of Apple AirDrop written in Python Poetry, a Python dependency management and packaging tool, releases v1 beta 1 with URL dependency Łukasz Langa at PyLondinium19: “If Python stays synonymous with CPython for too long, we’ll be in big trouble”

Microsoft yesterday unveiled Static TypeScript as an alternative to embedded interpreters. Static TypeScript (STS) is an implementation of a Static Compiler for TypeScript which runs in the web browser. It is primarily designed to aid school children in their computer science programming projects. STS is supported by a compiler that is also written in Typescript. It generates machine code that runs efficiently on Microcontrollers in the target RAM range of 16-256kB. Microsoft’s plan behind building Static TypeScript Microcontrollers are typically programmed in C, C++, or in assembly, none of which are particularly beginner friendly. MCUs that can run on modern languages such as JavaScript and Python usually involve interpreters like IoT.js, Duktape, or MicroPython. The problem with interpreters is high memory usage, leaving little room on the devices themselves for the program developers have written. Microsoft therefore decided to come with STS which is a more efficient alternative to the embedded interpreter approach. It is statically typed, which makes for a less surprising programming experience. Features of Static TypeScript STS eliminates most of the “bad parts” of JavaScript; following StrongScript, STS uses nominal typing for statically declared classes and supports efficient compilation of classes using classic techniques for vtables. The STS toolchain runs offline, once loaded into a web browser, without the need for a C/C++ compiler. The STS compiler generates efficient and compact machine code, which unlocks a range of application domains such as game programming for low resource devices . Deployment of STS user programs to embedded devices does not require app or device driver installation, just access to a web browser. The relatively simple compilation scheme for STS leads to surprisingly good performance on a collection of small JavaScript benchmarks, often comparable to advanced, state of the art JIT compilers like V8, with orders of magnitude smaller memory requirements. Differences with TypeScript In contrast to TypeScript, where all object types are bags of properties, STS has at runtime four kinds of unrelated object types: A dynamic map type has named (string-indexed) properties that can hold values of any type A function (closure) type A class type describes instances of a class, which are treated nominally, via an efficient runtime subtype check on each field/method access An array (collection) type STS Compiler and Runtime The STS compiler and toolchain (linker, etc.) are written solely in TypeScript. The source TypeScript program is processed by the regular TypeScript compiler to perform syntactic and semantic analysis, including type checking. The STS device runtime is mainly written in C++ and includes a bespoke garbage collector. The regular TypeScript compiler, the STS code generators, assembler, and linker are all implemented in TypeScript and run both in the web browser and on command line.  The STS toolchain, implemented in TypeScript, compiles STS to Thumb machine code and links this code against a pre-compiled C++ runtime in the browser, which is often the only available execution environment in schools. Static TypeScript is used in all MakeCode editors STS is the core language supported by Microsoft’s MakeCode Framework. MakeCode provides hands on computing education for students with projects. It enables the creation of custom programming experiences for MCU-based devices. Each MakeCode editor targets programming of a specific device or device class via STS. STS supports the concept of a package, a collection of STS, C++ and assembly files, that also can list other packages as dependencies. This capability has been used by third parties to extend the MakeCode editors, mainly to accommodate hardware peripherals for various boards. STS is also used in MakeCode Arcade. With Arcade, STS lets developers of all skill levels easily write cool retro-style pixelated games. The games are designed by the user to be run either inside a virtual game console in the browser or on inexpensive microcontroller-based handhelds. For more in-depth information, please read the research paper. People were quite interested in this development. A comment on Hacker News reads, “This looks very interesting. If all it takes is dropping “with, eval, and prototype inheritance” to get fast and efficient JS execution, I’m all for it.” Other news in tech TypeScript 3.6 releases with stricter generators, new functions in TypeScript playground, better Unicode support and more Microsoft announces XLOOKUP for Excel users that fixes most VLOOKUP issues Microsoft announces its support for bringing exFAT in the Linux kernel; open sources technical specs

Inspired by Google’s General Configuration Language (GCL), a team of developers has now come up with a new language called CUE. It is an open-source data validation language, which aims to simplify tasks that involve defining and using data. Its applications include data validation, data templating, configuration, querying, code generation and even scripting. There are two core aspects of CUE that set it apart from other programming or configuration languages. One, it considers types as values and second, these values are ordered into a lattice, a partially ordered set. Explaining the concept behind CUE the developers write, “CUE merges the notion of schema and data. The same CUE definition can simultaneously be used for validating data and act as a template to reduce boilerplate. Schema definition is enriched with fine-grained value definitions and default values. At the same time, data can be simplified by removing values implied by such detailed definitions. The merging of these two concepts enables many tasks to be handled in a principled way.” These two properties account for the various advantages CUE provides: Advantages of using CUE Improved typing capabilities Most configuration languages today focus mainly on reducing boilerplate and provide minimal typing support. CUE offers “expressive yet intuitive and compact” typing capabilities by unifying types and values. Enhanced readability It enhances readability by allowing the application of a single definition in one file to values in many other files. So, developers need not open various files to verify validity. Data validation You get a straightforward way to define and verify schema in the form of the ‘cue’ command-line tool. You can also use CUE constraints to verify document-oriented databases such as Mongo. Read also: MongoDB announces new cloud features, beta version of MongoDB Atlas Data Lake and MongoDB Atlas Full-Text Search and more! Easily validate backward compatibility With CUE, you can easily verify whether a newer version of the schema is backward compatible with an older one. CUE considers an API backward compatible if it subsumes the older one or if the old one is an instance of the new one. Allows combining constraints from different sources CUE is commutative, which means you can combine constraints from various sources such as base template, code, client policies, and that too in any order. Allows normalization of data definitions Combining constraints from many resources can also result in a lot of redundancy. CUE’s logical inference engine addresses this by automatically reducing constraints. Its API allows computing and selecting between different normal forms to optimize for a certain representation. Code generation and extraction Currently, CUE can extract definitions from Go code and Protobuf definitions. It facilitates the use of existing sources or smoother transition to CUE by allowing the annotation of existing sources with CUE expressions. Querying data CUE constraints can be used to find patterns in data. You can perform more elaborate querying by using a ‘find’ or ‘query’ subcommand. You can also query data programmatically using the CUE API. On a Hacker News discussion about CUE, many developers compared it with Jsonnet, which a data templating language. A user wrote, “It looks like an alternative to Jsonnet which has schema validation & strict types. IMO, Jsonnet syntax is much simpler, it already has integration with IDEs such as VSCode and Intellij and it has enough traction already. Cue seems like an e2e solution so it's not only an alternative to Jsonnet, it also removes the need of JSON Schema, OpenAPI, etc. so given that it's a 5 months old project, still has too much time to evolve and mature.” Another user added, “CUE improves in Jsonnet in primarily two areas, I think: Making composition better (it's order-independent and therefore consistent), and adding schemas. Both Jsonnet and CUE have their origin in GCL internally at Google. Jsonnet is basically GCL, as I understand it. But CUE is a whole new thing.” Others also praised its features. “When you consider the use of this language within a distributed system it's pretty freaking brilliant,” a user commented. Another user added, “I feel like that validation feature could theoretically save a lot of people that occasional 1 hour of their time that was wasted because of a typo in a config file leading to a cryptic error message.” Read more about CUE and its concepts in detail, on its official website. Other news in Programming languages ‘Npm install funding’, an experiment to sustain open-source projects with ads on the CLI terminal faces community backlash “Rust is the future of systems programming, C is the new Assembly”: Intel principal engineer, Josh Triplett Kotlin 1.3.50 released with ‘duration and time Measurement’ API preview, Dukat for npm dependencies, and much more!

Last week, the Golang team announced that the Go module mirror, index, and checksum database are now production-ready thus adding reliability and security to the Go ecosystem. For Go 1.13 module users, the go command will use the module mirror and checksum database by default. New production-ready modules for Go 1.13 module Module Mirror A module mirror is a special kind of module proxy that caches metadata and source code in its own storage system. This allows the mirror to continue to serve source code that is no longer available from the original locations thus speeding up downloads and protect users from the disappearing dependencies. According to the team, module mirror is served at proxy.golang.org, which the go command will use by default for module users as of Go 1.13. For users still running an earlier version of the go command, they can use this service by setting GOPROXY=https://proxy.golang.org in their local environment. Read Also: The Go team shares new proposals planned to be implemented in Go 1.13 and 1.14 Module Index The module index is served by index.golang.org. It is a public feed of new module versions that become available through proxy.golang.org. Module index is useful for tool developers who want to keep their own cache of what’s available in proxy.golang.org, or to keep up-to-date on some of the newest modules go developers use. Read Also: Implementing Garbage collection algorithms in Golang [Tutorial] Checksum Database Modules introduced the go.sum file, a list of SHA-256 hashes of the source code and go.mod files of each dependency when it was first downloaded. The go command can use these hashes to detect misbehavior by an origin server or proxy that gives a different code for the same version. However, the go.sum file has a limitation, it works entirely by trust based on user’s first use. When a user adds a version of a never seen before dependency, the go command fetches the code and adds lines to the go.sum file quickly. The problem is that those go.sum lines aren’t being checked against anyone else’s and thus they might be different from the go.sum lines that the go command just generated for someone else. The checksum database ensures that the go command always adds the same lines to everyone's go.sum file. Whenever the go command receives new source code, it can verify the hash of that code against this global database to make sure the hashes match, ensuring that everyone is using the same code for a given version. The checksum database is served by sum.golang.org and is built on a Transparent Log (or “Merkle tree”) of hashes backed by Trillian, a transparent, highly scalable and cryptographically verifiable data store. The main advantage of a Merkle tree is that it is tamper-proof and has properties that don’t allow for misbehavior to go undetected, making it more trustworthy. The Merkle tree checks inclusion proofs (if a specific record exists in the log) and “consistency” proofs (that the tree hasn’t been tampered with) before adding new go.sum lines to a user’s module’s go.sum file. This checksum database allows the go command to safely use an otherwise untrusted proxy. Because there is an auditable security layer sitting on top of it, a proxy or origin server can’t intentionally, arbitrarily, or accidentally start giving you the wrong code without getting caught. “Even the author of a module can’t move their tags around or otherwise change the bits associated with a specific version from one day to the next without the change being detected,” the blog mentions. Developers are excited about the launch of the module mirror and checksum database and look forward to checking it out. https://twitter.com/hasdid/status/1167795923944124416 https://twitter.com/jedisct1/status/1167183027283353601 To know more about this news in detail, read the official blog post. Other news in Programming Why Perl 6 is considering a name change? The Julia team shares its finalized release process with the community TypeScript 3.6 releases with stricter generators, new functions in TypeScript playground, better Unicode support for identifiers and more