Monday, January 23, 2017

JDK 9 is the End of the Road for Some Features

It was announced a few days ago that JDK 9 is Feature Complete! Many of the "features" that made the cut are additions, but some are removals. This post looks at some of the items being removed from OpenJDK and/or Oracle's JDK with Java 9.

Endorsed-standard Override Mechanism and Extension Mechanism

Part of JEP 220 ("Modular Run-Time Images") is the removal of the Java Endorsed Standards Override Mechanism ("means whereby later versions of classes and interfaces that implement Endorsed Standards or Standalone Technologies may be incorporated into the Java Platform") and the removal of the Extension Mechanism for Support of Optional Packages ("allows the Java virtual machine (VM) to use the classes of the optional extension in much the same way as the VM uses classes in the Java Platform"). Upgradeable Modules are intended to replace the endorsed-standard override mechanism in JDK 9. Regardung the extension mechanism, JEP 220 states, "The extension mechanism was introduced in JDK 1.2, which was released in 1998, but in modern times we have seen little evidence of its use." Erik Costlow's post Planning safe removal of under-used 'endorsed extension' directories provides more background on these removals and how to detect if they will affect a particular Java application.

rt.jar and tools.jar

JEP 220 also removes the well-known and frequently-referenced JARs rt.jar and tools.jar. JEP 220 explains, "The class and resource files previously stored in lib/rt.jar, lib/tools.jar, lib/dt.jar, and various other internal jar files will now be stored in a more efficient format in implementation-specific files in the lib directory. The format of these files will not be specified and is subject to change without notice." Additional details on the removal of these JARs and anticipated effects of these removals can be found in Java 9 completely changes JDK directory layout and drops tools.jar, How Java 9 And Project Jigsaw May Break Your Code, and Prepare for JDK 9. No one argues that it was well advertised that these APIs were not intended for external use, but "some popular libraries make use of non-standard, unstable, and unsupported APIs that are internal implementation details of the JDK and were never intended for external use."

"Most" Internal APIs (But Not sun.misc.Unsafe!)

JEP 260 ("Encapsulate Most Internal APIs") was written to "make most of the JDK's internal APIs inaccessible by default [in JDK 9] but leave a few critical, widely-used internal APIs accessible, until supported replacements exist for all or most of their functionality." Mark Reinhold's message Encapsulating internal APIs in JDK 9 (sun.misc.Unsafe, etc.) describes the motivation for this in detail. This approach was a compromise after the uproar over removing the internal (and infamous) API sun.misc.Unsafe. The JEP 260 documentation discusses in some detail which internal APIs will remain accessible in JDK 9 and explains that some will be deprecated in JDK 9 and removed altogether at some later point. The "critical internal APIs proposed to remain accessible in JDK 9" include sun.misc.Unsafe, sun.misc.Signal, and sun.misc.SignalHandler.

Java DB

In the post Deferring to Derby in JDK 9, Oracle's Don Smith writes, "Java DB is simply a re-branded distribution of the Apache Derby open source database. It contains the same binaries as Apache Derby. As of JDK 9, it is no longer being planned to include it in the 'db' directory of Oracle JDK downloads. Developers looking ahead to JDK 9 should plan to get and bundle Apache Derby independently for the same purpose." As of this writing, the Oracle Java DB page states, "Java DB is Oracle's supported distribution of the Apache Derby open source database. It supports standard ANSI/ISO SQL through the JDBC and Java EE APIs. Java DB is included in the JDK." This will no longer be the case as of JDK 9.

Garbage Collector Options Deprecated Since JDK 8

As documented in the "Removed" section of the OpenJDK page JDK 9 Outreach and in JEP 214, "rarely used" garbage collection options combinations deprecated in JDK 8 are now removed altogether (meaning that the inclusion of these combinations of options will prevent JVM from starting rather than simple presenting warning messages). Here are the combinations of garbage collection options to be removed:

DefNew + CMS-XX:-UseParNewGC -XX:+UseConcMarkSweepGC
ParNew + SerialOld-XX:+UseParNewGC
ParNew + iCMS-Xincgc
ParNew + iCMS-XX:+CMSIncrementalMode -XX:+UseConcMarkSweepGC
DefNew + iCMS-XX:+CMSIncrementalMode -XX:+UseConcMarkSweepGC -XX:-UseParNewGC
CMS foreground-XX:+UseCMSCompactAtFullCollection
CMS foreground-XX:+CMSFullGCsBeforeCompaction
CMS foreground-XX:+UseCMSCollectionPassing

jhat

JEP 241 is called "Remove the jhat Tool" and its concise "Summary" is, "Remove the antiquated jhat tool." The motivation to remove jhat is explained in JDK-8059039, "jhat was added in JDK 6, based upon the java.net HAT project. jhat is an experimental, unsupported, and out-of-date tool. Superior heap visualizers and analyzers have now been available for many years." Alternatives to jhat include Eclipse Memory Analyzer Tool (MAT) and VisualVM. This removal is also documented in the OpenJDK JDK 9 Outreach documentation and is mentioned in the article OpenJDK 9: Life Without HPROF and jhat.

Java Virtual Machine Tool Interface hprof Agent

JEP 240 removes the JVM TI hprof Agent from the JDK. The JDK 8 TechNotes on HPROF: A Heap/CPU Profiling Tool states (I've added the emphasis), "The Java 2 Platform Standard Edition (J2SE) has always provided a simple command line profiling tool called HPROF for heap and cpu profiling. HPROF is actually a JVM native agent library which is dynamically loaded through a command line option, at JVM startup, and becomes part of the JVM process." As stated in JDK-8046661 used to remove this, there are other alternatives available for generating "heap dumps in the hprof format" including jmap -dump and jcmd <pid> GC.heap_dump. This is also referenced on the JDK 9 Outreach page and is discussed in the article OpenJDK 9: Life Without HPROF and jhat.

VisualVM

Oracle's Aurelio Garcia-Ribeyro writes in the post Visual VM in JDK 9 and Beyond that "starting with JDK 9, Visual VM will not be included with Oracle JDK" and adds, "Developers who would like to use Visual VM with Oracle JDK 9 or later can get it from the Visual VM open source project site. This appears to be correlated with Oracle's earlier decision to donate NetBeans to Apache Software Foundation (VisualVM is based on the NetBeans Platform). One of the primary tools to use instead of the removed jhat tool and HPROF agent will also need to be downloaded separately for use with JDK 9.

AppleScript Engine

AppleScript engine code is removed with JDK 9 and this removal is documented in the "Removed" section of the OpenJDK page JDK 9 Outreach.

HTTP Proxying from RMI

HTTP proxying from RMI was deprecated with JDK 8 and is being removed from JDK 9. This is called out on the JDK 9 Outreach page.

Default Resolution of java.corba and EE Modules

The java.corba and other EE modules will still be available in JDK 9, but they will no longer be visible by default. The six modules "that will not be visible by default" are java.activation, java.annotations.common, java.corba [deprecated], java.transaction, java.xml.bind, and java.xml.ws. JEP 261 describes this in more detail and explains, "Modules that define Java EE APIs, or APIs primarily of interest to Java EE applications, are not resolved by default for code on the class path." It calls this change an "intentional, if painful, choice" intended "to avoid unnecessary conflicts with popular libraries that define types in some of the same packages" and "to make it easier for existing application servers to migrate to JDK 9.".

-source and -target Support for Java 1.5

The "Summary" of JEP 182 ("Policy for Retiring javac -source and -target Options") states, "To reduce the maintenance costs of javac, this JEP defines a policy for retiring old -source and -target options. In JDK 8, use of a source or target of 1.5 or earlier will be deprecated and in JDK 9, support for a source or target of 1.5 or earlier will be removed. In JDK 9 and going forward, javac will use a 'one + three back' policy of supported source and target options. Under this policy, javac will still be able to recognize and process class files of all previous JDKs, going back to version 45.3 class files generated by JDK 1.0.2, which first shipped in 1996."

Per the policy outlined in JEP 182, JDK 9 "remove[s] support for 1.5 and earlier source and target options" via JDK-8011044. Joe Darcy described this change when it was delivered, "The javac command no longer support -source or -target options below 6/1.6. However, older class files are still readable by javac. Source code for older release can be ported to the newer source level. To generate class files usable by older release, a javac from a previous release train can be used."

Other Removals

The "Removed" section of the OpenJDK JDK 9 Outreach page briefly references some of the items I discussed in this post and also references some items being removed from JDK 9 that I did not reference in this post:

  • JEP 231 ("Remove Launch-Time JRE Version Selection"): "Remove the ability to request, at JRE launch time, a version of the JRE that is not the JRE being launched."
  • JDK-8037739: "Remove API references to java.awt.peer and java.awt.dnd.peer packages in the JDK 9 timeframe"
  • JDK-8029806: "Remove Packer/Unpacker addPropertyChangeListener and removePropertyListener methods"
  • JDK-8029805: "Remove LogManager addPropertyChangeListener and removePropertyChangeListener methods"
  • JDK-8029904: ""Remove com.sun.security.auth.callback.DialogCallbackHandler"
  • JDK-7067728: "Remove stopThread RuntimePermission from the default java.policy"
  • JDK-8134808: "Remove support for serialized applets from java.desktop"

Conclusion

Many of the items being removed from JDK 9 are being removed because of changes to the JDK to support modularity (JEP 200). Other items are being removed because there are better supported alternatives or because the products formerly included with the JDK are now expected to be downloaded separately.

Thursday, January 19, 2017

JDK 9 is Feature Complete!

Today's Mark Reinhold message JDK 9 is Feature Complete -- now it's time to ramp down announces that JDK 9's "overall feature set is, at this point, frozen." Reinhold, Chief Architect of Oracle's Java Platform Group, adds, "It's highly unlikely that any further JEPs will be targeted to the release." Text similar to that in this message is also featured on the JDK 9 page (which was not coincidentally updated today).

The JDK 9 page lists nearly ninety features, including the following subset listed here:

According to Reinhold's message and the current JDK 9 page, the JDK 9 Rampdown Phase has begun "in which we aim to fix the bugs that need to be fixed and understand why we're not going to fix some bugs that perhaps ought to be fixed."

Reinhold and the JDK 9 page state that "it is highly unlikely that any further JEPs will be targeted to the [JDK 9] release." They add the types of things that might still be added:

  • "Small enhancements to new features will be considered, but the bar is now much higher."
  • "Low-risk enhancements that add small bits of missing functionality or improve usability may be approved, especially when justified by developer feedback."
  • "Enhancements to tests or documentation do not require advance approval."

The JDK 9 page currently shows 27 July 2017 as the "General Availability" date for JDK 9.

Saturday, December 31, 2016

Significant Software Development Developments of 2016

This post is my personal and opinionated assessment of some of the most significant developments related to software development in 2016. This is my tenth year for this annual post and my previous years' assessment are available for 2015, 2014, 2013, 2012, 2011, 2010, 2009, 2008, and 2007. As with these previous years' assessments, this assessment of 2016's major developments in software development are obviously biased, opinionated, and limited to my perspective.

10. Tumultuous Year for Java

Java and the JVM are used by a significant percentage of software developers worldwide. With the increasing adoption of Java 8 and the enthusiasm Java 8 has helped to reinvigorate among Java developers, Java SE and the JVM seem well positioned for the future. Unfortunately, Java EE faced more issues in 2016. 2016 saw the rise of the Java EE Guardians in response to a lack of news from Oracle regarding the future of Java EE. Oracle eventually did respond in advance of JavaOne 2016 with plans for Java EE 8 being more cloud and microservices oriented. Oracle has also stated that their survey of Java EE developers has shown that REST and HTTP/2 support are of prime importance.

Even Java SE did not survive 2016 unscathed. In this last month of 2016, an article appeared in The Register called Oracle finally targets Java non-payers - six years after plucking Sun stated, "Java SE can be free, although Java SE Advanced Desktop, Advanced and Suite are not. ... There's also no way to separate the paid-for Java SE component products from the free Java SE umbrella at download as Oracle doesn't offer separate installations – they're all bundled together and that leads to confusion and mistakes down the line when paid-for features are unexpectedly used." Several articles and posts have tried to add clarifying details, but all of these together do leave the impression that it's a licensing situation that is more difficult than it should be. Other posts on the subject include Java SE Offerings, The Sky Is Falling: Oracle (Might) Want Your Money for Java SE in 2017, and Java Champions response to The Register Article on Java SE usage.

See the post Goodbye 2016 (Hello 2017) for more details on Java in 2016. Another year-end discussion surrounding Java is available in the OverOps's blog post If You've Written Java Code in 2016 - Here Are the Trends You Couldn't Have Missed. Another OverOps's Takipi blog post on Java in 2016 is Java Community Oscars - The Top 10 Posts of 2016 (published since the publication of my post).

9. DevOps

Although the DevOps concept has been around for a while now, it's gained popularity with the "suits" (management) and the term is thrown around in all types of contexts. Software product vendors emphasize how their products continue to DevOps and software development requisitions and job listings are filled with references to experience with DevOps.

8. Angular 2

Angular 2 final was released in 2016 and was a significant re-write of the already popular Angular framework. The Angular 1 TO 2 Quick Reference documents "many conceptual and syntactical differences between Angular 1 and Angular 2." The article Angular 2 and TypeScript - A High Level Overview points out that Angular 2 was re-written in Microsoft's TypeScript language and that this is the "recommended language for developing applications using the [Angular] framework" (although Angular-based applications can also be written in JavaScript and Dart). There are many things to like about TypeScript, especially for those of us with more Java experience than JavaScript experience.

The Angular community also provided a react-native-renderer library in 2016 for using "Angular 2 and React Native to build applications for Android and iOS." There is also now a dedicated AngularDart team that supports the Dart implementation of Angular that is, as of 2016, separate from the TypeScript/JavaScript code base. Before this, the Angular 2 codebase was entirely TypeScript and was compiled to Dart.

Angular news in 2016 included the adoption of semantic versioning for releases and the releases of Angular 1.6.0, Angular 1.5.0, Angular 2.4.0, Angular 2.3.0, Angular 2.2.0, Angular 2.1.0, the final release of Angular 2 (2.0.0), and several Angular 2 release candidates (RC7, RC6, RC5, RC4, RC3, and RC2).

Angular continued to enjoy adoption and coverage in 2016 and it already looks like 2017 will be a big year for Angular with the announced release of Angular 4 (skipping an Angular 3).

7. Internet of Things (IoT)

The term "Internet of Things" (IoT) has been around for a while and even discussed in mainstream media for a while, but the concept seems to come up in many different technology-related articles and posts in 2016. Because the IoT concept is related to Big Data and Cloud Computing, it is not surprising that it becomes a more prevalent topic as those related topics become increasingly prevalent. The rise of IoT also brings with it significant new privacy and security risks while also bringing convenience to users.

The post IoT 2016 in review: The 8 most relevant IoT developments of the year talks about the increasing presence of consumer IoT with products such as Amazon Echo. This article also discusses the "biggest overall [IoT] story" of 2016: "IoT Security attack on Dyn servers," a story which is part of my reasoning for Software Security being the #1 story/development of 2016 for software developers.

The CRN slideshow The 10 Biggest Internet Of Things Stories Of 2016 states that "2016 was an even bigger year for IoT" and outlines their perspective on the biggest IoT stories on 2016. Their top story for the year is IoT Devices Compromised In DDoS Attack.

6. Microsoft and Open Source

The Linux Foundation's announcement in November that "Microsoft has joined the [Linux Foundation] [as] a Platinum member" would have been unfathomable a decade ago, but is less surprising in 2016 given Microsoft's open source contributions that include Ubuntu on Windows 10, PowerShell (Linux and Macs), Xamarin SDK, TypeScript, and numerous other products, libraries, and frameworks available on the Microsoft GitHub repositories.

Other Microsoft-related announcements in 2016 include SQL Server running on Linux, Microsoft joining the Eclipse Foundation, and the release of Visual Studio Code 1.0 for Linux. Microsoft also announced Windows Server 2016 this year, which features support for Windows Containers (Docker-based; Microsoft has a commercial partnership with Docker). Microsoft also brought support for React Native to Windows (Universal Windows Platform).

5. Big Data

Big Data remains, well, "big," in 2016. In 2016: The Year in Big Data, Tim Spann writes that "2016 has been an interesting year" in which "Big Data got big general interest when various machine learning algorithms were predicting who may become president (and were a bit off)." Spann also writes that 2016 was a year in which Big Data experienced "new advances in tools, an expanded focus on IoT, and new ways of ingesting and manipulating data." The post 5 Leading Developer Stories of 2016 emphasizes, "Big Data's Big Numbers Became Impossible to Ignore in 2016.

4. Cloud Computing

Cloud Computing remained a hot topic in 2016. In the article Why the cloud? In 2016, it was the lure of the new, Eric Knorr writes that "incredible new features all pre-provisioned and waiting for you in the cloud" was the primary reason for enterprises moving to the cloud in 2016. Knorr writes about the cloud offerings' support for machine learning, IoT platforms, serverless computing, and containers management. The IBM blog post The top 10 Thoughts on Cloud articles of 2016, states, "Much of the cloud computing news of 2016 focused on exponential growth in need and demand, which will reach into the following year and beyond."

Related to both Cloud Computing and Big Data is the concept of Fog Computing, which also seems to be slowly gaining momentum in 2016.

3. Microservices

Although I see "Docker", "containers", "Big Data", "Cloud Computing" and similar terms all over the software development blogosphere, I don't think there's a more prevalent topic in that sphere than "microservices." Bilgin Ibryam has written that microservices are a commodity and concludes, "In the Microservices world things are moving from uncharted to industrialized direction. Most of the activities are not that chaotic, uncertain and unpredictable. It is almost turning into a boring and dull activity to plan, design and implement Microservices. And since it is an industrialized job with a low margin, the choice of tool, and ability to swap those platforms plays a significant role."

The microservices architecture approach has been used by enough organizations over enough time now to start sharing lessons learned and best practices. In fact, many of the underlying ideas behind microservices have existed for much longer than the term itself. In the post Working with microservices framework still a challenge in 2016, Tom Nolle writes, "Technology innovation is often hampered by fast-moving waves of hype that confuse senior management with benefit claims that can't be met early in a technology's evolution. Microservices really took off as a concept in 2016 and have been vulnerable to premature expectations." Knoll writes about some positive developments in microservices that helped mitigate this: association with private clouds and with containers and the move toward statelessness in microservices. Knoll also writes about misuse of microservices contributing to a negative experience with them.

Some signs of the increasingly prevalent use of microservices include many corporations adopting the term for their products and services. Other signs include the numerous posts for (mostly) and against (blind) adoption of microservices. These very recently published posts include How can a big enterprise benefit from microservices?, Microservices: Good for Customers, Good for Business, Why Isn't Microservices for Everyone?, Maybe beware of microservices, Why you should adopt a microservices architecture, and Microservices: Not Just for Nerds Anymore.

2. Docker and Containers

Without putting much effort into verifying it, it felt like I could not browse software development headlines on related social media sites such as DZone without seeing at least one (and often many more) story on Docker or other Linux containers. Serdar Yegulalp's InfoWorld article 2016: When Linux containers became mainstream states, "In truth, 2016 wasn't The Year of the Container. That was 2015, when the possibility and promise of containers came along and knocked IT for a loop. But not everyone wanted creative disruption. Many folks wanted dependable, reliable infrastructure, and they saw in containers a method to do so that had never been done before. The good news was that despite all the momentum around containers in 2016, major parts of the ecosystem began to stabilize. The novelty's worn off, but in a good way - it means there's now more attention on how to do containers right, not merely to do them at all."

I have already written in this post about Microsoft's partnership with Docker. Just this month, Docker announced plans to provide its open source "Core Container Runtime" (containerd) to an open source foundation (changing who controls the open source project). Docker Engine releases in 2016 include 1.13.0, 1.12.3, 1.12.2, 1.12.1, 1.12.0 (including built-in orchestration), 1.11.1, 1.11.0, 1.10.3, 1.10.2, 1.10.1, and 1.10.0.

Since my original post, Top Docker Content Of 2016 has been published.

1. Software Security and Software Outages

I really, really wanted to not have this be the #1 software development in 2016 after having a version of this in the top two spots in recent years (2012, 2013, 2014, and 2015). Unfortunately, the events of 2016 made it impossible for me to bump software security related issues from the #1 spot of developments in software development.

The Norton Protection Blog features the post Data Breaches That Made Headlines in 2016. This post references and summarizes several data breaches including the 2012 LinkedIn breach (117 million released sensitive credentials in 2016), 2012 DropBox breach (68 million credentials exposed in 2016), and the 2013 Yahoo! breach (1 billion accounts breached). In the post Yahoo Announces Breach of One Billion Accounts, Nadia Kovacs writes, "2016 seems to be the year of the 'mega-breach' with us reporting on eight major breaches involving well-known companies. Big data is big money for attackers, so they set their sights on companies that tend to hold large amounts of personally identifiable data on their customers." It's interesting that the three large breaches I referenced actually occurred in previous years, but the magnitude of the breaches was discovered with release of the sensitive data in 2016.

Even the products intended to aid in efforts to secure software and data had their own vulnerabilities exposed in 2016. For example, Google Found Disastrous Symantec and Norton Vulnerabilities That Are 'As Bad As It Gets', Kernel Information Leak & Multiple DOS Issues Within Kaspersky Internet Security Suite, and vulnerabilities associated with a system running Intel's McAfee VirusScan Enterprise for Linux.

Issues related to software security played a role in the already strange presidential election in the United States in 2016. Allegations of mishandling of e-mail messages that were potentially made more vulnerable by being on the candidate's own server led to an FBI investigation and pestered a candidate throughout the election. The hacking and exposing of sensitive and sometimes highly opinionated e-mail messages of Democratic National Committee members with some "uncomfortable disclosures" also played out during the election year. There have also been allegations of large-scale hacking of the votes counting. The article US recounts find no evidence of hacking in Trump win but reveal vulnerabilities states that this probably did not occur, but warns that "we found that hacking an election in the US for president would be even easier than I thought." It has also been suggested that cyber attacks were levied against previous elections.

On top of all this, increased hacking of mobile telephones and devices continued with stories such as Android bug fear in 900 million phones. Many end-users were impacted worldwide by the network outages and network congestion caused at least in part to hacked Internet of Things (IoT) devices. Bruce Schneier has written this year that Someone Is Learning How to Take Down the Internet.

The CRN slideshow The 10 Biggest Security Stories Of 2016 starts on the first slide, "Every year, cybersecurity continues to play an even bigger role in both business and consumer decisions. In 2016, new issues around the election, public-private tug of wars over encryption technologies and major attacks brought security even further into the forefront than in years past." That slideshow also references the U.S. presidential election, the Yahoo! breach, and the distributed denial of service (DDoS) attack on Dyn as most significant security stories in 2016.

Fahmida Y. Rashid's recent InfoWorld article asks, "Do developers really care about security?" We obviously need to care. I would love to see Software Security not be my #1 development in software development in 2017 and not even be in my top ten, but it's difficult for me to see that happening.

Honorable Mention

The following did not make my top ten, but nevertheless saw major developments in 2016.

Apache NetBeans

Oracle has turned over the freely available open source Java IDE NetBeans to the Apache Software Foundation and NetBeans is now in the Apache Incubator.

Javaslang

Interest in Javaslang seems to have really take off in 2016, probably at least in part due to the increased adoption of Java 8 in 2016. Javaslang is a "functional library for Java 8+" that aims to "reduce the amount of code" and increase "robustness." It provides an Option that is richer than Java 8's Optional and provides "immutable collections and the necessary functions and control structures to operate on these values." A nice introduction to Javalang can be found in Eugen Paraschiv's Baeldung post Introduction to Javalang.

Kotlin

There's no shortage of programming languages targeting the Java Virtual Machine as their runtime and some of them (Scala, Groovy, Clojure, Ceylon etc.) seem to have found success with JVM developers. Without any empirical basis for it, my feeling is that Kotlin may have enjoyed the most coverage and percentage rise in adoption in 2016. This could just be perspective, but JetBrains' purchasing of advertisements on key software development web sites and evangelism of its employees and fans/customers seem to have contributed to Kotlin being mentioned time and time again on the software development sites (especially those with JVM focus).

Kotlin's main page describes it as a "statically typed programming language for the JVM, Android and the browser" that is "100% interoperable with Java." Kotlin 1.0 was released in February of this year and 2016 has seen later releases of Kotlin (1.0.6, 1.0.5, 1.0.4, 1.0.3, 1.0.2, 1.0.1, and several 1.1 milestone releases).

Gradle Script Kotlin was introduced this year and it was announced that Kotlin scripting support would be included with Gradle 3.0 (see status update). The driving factor for Gradle transitioning from Groovy to Kotlin as the primary build script language appears to be the ability of IDEs to better support a statically typed language. A Kotlin NetBeans plugin was also released in 2016.

Since my publishing of this post, it was announced that Spring 5 will support Kotlin.

Gradle

2016 saw the release of Gradle 3.0 as well as the introduction of Gradle Script Kotlin. Gradle Build Scans (cloud-based "shareable and centralized record of a build that provides insights into what happened and why") were also announced at the 2016 Gradle Summit as the first offering of Gradle Cloud Services.

Scala

Scala 2.12.0 (and Scala 2.12.1) was released in 2016 with the Scala 2.12 compiler "completely overhauled to make use of the new VM features available in Java 8." Scala 2.12 requires JVM 8 runtime because "Scala 2.12 is all about making optimal use of Java 8's new features." A less rosy outlook on Scala 2.12 can be found in the post New Scala Release Falls flat.

The Scala Center at EPFL was announced in 2016 as "an open source foundation for Scala" that "will engage with developers in the open-source community in the effort to improve the language, its tools and its documentation" and "will also help developers learn the language through massive open online courses, seminars and other activities."

PHP

Although PHP 7 was technically released in late 2015, it is 2016 that began to see significant adoption of PHP 7. Prateek Tiwari has writes that PHP 7 brings "a new revolution in web development" and Zend's page states, "PHP 7 makes powering the web a whole lot better."

C++

I don't actively develop with C++ anymore, but do find it interesting to see what's happening in that still very active world from time-to-time. Bartlomiej Filipek has posted C++ Status at the end of 2016.

Fragility of Open Source

Like me, most developers I know are generally positive about open source and the possibilities it enables. We have benefited numerous times from a wide variety of open source products. However, nothing is perfect and open source has its own issues. We were reminded of this somewhat dramatically in 2016 when Azer KoƧulu liberated all of his modules from NPM after a trademark dispute over kik (now known as hek). It was actually unpublishing of left-pad from NPM that caused the ruckus.

There was controversy surrounding the widely used FindBugs in 2016 as well. In the post Where FindBugs Failed, Nicolai Parlog writes that some of the contributors to FindBugs forked the project as SpotBugs. As of this writing, the main FindBugs page states, "The current version of FindBugs is 3.0.1, released on 13:05:33 EST, 06 March, 2015."

Simon Phipps's article Uproar: MariaDB Corp. veers away from open source talks about MariaDB taking MaxScale from open source to proprietary via a "Business Source License."

One interesting note about these stories related to open source issues is that each of these has been addressed through steps that take advantage of the code in question being open source. Because these projects were open source, others in the community were able to fork the projects when they did not like their direction or replace the essential code when removed code broke things.

StackOverflow Documentation

StackOverflow Documentation premiered in 2016 and is described as "community-curated, example-focused developer documentation, based on the principles of Stack Overflow." As I wrote in my post Will StackOverflow Documentation Realize Its Lofty Goal?, time will tell if this will have anywhere near the effect on software development that StackOverflow has had.

C# / .NET

.NET Core was released in 2016. The .NET Blog post Announcing .NET Core 1.0 describes .NET Core as "a cross-platform, open source, and modular .NET platform for creating modern web apps, microservices, libraries and console applications" that is "available on Windows, OS X and Linux!"

There is a decent chance that C# 7.0 will place C# on my Top Ten list at this time next year. Many of the features of the forthcoming C# 7.0 are available now in Visual Studio "15" Preview 4, released in August 2016. Numerous articles in late 2016 have pointed to C# 7.0 and include What's New in C# 7.0, C# 7 - What to Expect, and New Features in C# 7. VisualStudio 2017 is also forthcoming.

Python

Python 3.6.0 was released in late 2016 as was Python 2.7.13. The Python repository moved to GitHub instead of GitLab in 2016. Much more detailed information on Python in 2016 can be found in Dibya Chakravorty's post Python year in review 2016. I have been watching from time to time the debate about Python 2.x versus Python 3.x in attempt to learn lessons from breaking backwards compatibility and found it interesting in Chakravorty's post that the following was stated, "The debate between Python 2 and 3 is never ending, but the PSF had declared that Python 2.7 will reach end of life in 2020."

Responsive Web Design

The concept of "responsive web design" has been around for a while, but seemed to gain even more traction in 2016, perhaps in part because of Google's efforts to use it and evangelize it. Although 2013 has been declared the year of responsive web design, the years since seem to have continued to see it as a popular concept for dealing with multiple client types.

Progressive Web Apps

There's been a lot of buzz out of Google about progressive web apps. The author of the post What are Progressive Web Apps? opens that post with the only slightly exaggerated, "By now, you’ve probably heard that Progressive Web Apps are the future of all mankind. They’ll bring world peace, end hunger, save the rainbows and unicorns, bring balance to the force, and a whole lot more. In the process, they might even push the mobile web forward, bring parity to web and native apps, and help mobile developers reach more users beyond the confines of the app stores." That same post provides more clarification of progressive web apps, "On the whole, Progressive Web Apps describe a collection of technologies, design concepts, and Web APIs that work in tandem to provide an app-like experience on the mobile web."

The term "progressive web apps" was coined in 2015 and has rapidly generated excitement in the web app development community in 2016.

JavaScript

JavaScript continues to be a widely used language, in large part because of its ability to run in web browsers (and on mobile devices) and on the server. Craig Buckler has written in the post JavaScript: 2016 in Review and concludes, "it's increasingly difficult to keep pace with the latest trends, frameworks and recommendations. Developers struggle when faced with a plethora of options to evaluate. My advice: don’t try to keep up. It's impossible. Whatever system you bet on today will be superseded by something better tomorrow. ... There's only one absolute certainty: JavaScript itself. Learn the language first and keep building on your knowledge. Your experience will help you understand how each framework operates so you can make an informed choice. That choice may be to forego frameworks altogether." This quote tells how far JavaScript has come; the last thing I would have wanted to do when starting with JavaScript many years ago was to use it without a framework to hide all of the idiosyncrasies and corners of the browser DOMs.

JS Foundation

The JS Foundation was formed in 2016 as a Linux Foundation Project "to drive broad adoption and ongoing development of key JavaScript solutions and related technologies." In an example of rebranding (see item below for other rebranding efforts in 2016), the JS Foundation was created from the jQuery Foundation.

React and React Native

Although JavaScript, like Java and other programming languages, seems to have countless frameworks, React seems to me to be the one getting the most attention in the JavaScript community when not counting Angular. The project facebook/react currently has a very slight edge over angular/angular.js in terms of stars rating on GitHub.

In April of this year, the Facebook post React Native: A year in review celebrated "one year since [Facebook] open-sourced React Native." The post Facebook Open Source 2016 year in review celebrates the rapid adoption of React and Native React.

In December 2016, Oculus Connect 3 introduced "the React VR Pre-Release" based on React and React Native.

Software Development Tools Companies' Rebranding

2016 seems to be another Year of Rebranding for many software development tools makers and continues the trend from 2015:

Rust

Although Rust 1.0 was released in mid-2015, 2016 was another year in which it saw increased popularity and made many "trendy" languages lists. The Rust programming language is defined on its home page as "a systems programming language that runs blazingly fast, prevents segfaults, and guarantees thread safety." Its most recent version, 1.14.0, was released just days ago in late 2016, but the year also saw several other releases: 1.13.0, 1.12.1, 1.12.0, 1.11.0, 1.10.0, 1.9.0, 1.8.0, 1.7.0, and 1.6.0 (releases 1.0.0 through 1.5.0 were released in 2015). The "preferred tool for installing Rust" (rustup 1.0.0) was released in late 2016.

One of the key highlights of the "Stack Overflow Developer Survey Results 2016" is, "Developers love Rust." The survey found Rust to rank highest of the programming languages in terms of "most loved" with 79.1%. The Charlie Crawford post How Rust Compares to Other Programming Languages states that Rust is "safer than C/C++" and "more sophisticated than Go." Rachel Roumeliotis writes in Hot programming trends in 2016 that Rust is one of "five languages are on the verge of making it into the big time [this year]." Rust has been named one of Davide Aversa's Most Promising Programming Languages of 2017 and Kyle Prinsloo recommends it as one of the Top Programming Languages to Learn in 2017.

The State of Rust Survey 2016 highlights some of Rust's wins and some things that need to be improved. A book on Rust was published in 2015 and two more are scheduled for 2017. The O'Reilly e-book Why Rust? can currently be downloaded in PDF.

Swift

2016 was a big year for the Swift programming language with the biggest news perhaps being the release of Swift 3.0 (noted as the "first major release of Swift since it was open-sourced", which was one of Swift's major 2015 happenings). Reddit user lkraider describes Swift 3 as "basically the first usable version of the language, applicable from mobile to backend dev." Ted Neward has written about the struggles of updating from Swift 1 to Swift 2 to Swift 3 due to backwards source incompatibility.

Swift was the second "most loved" language (after Rust) in the StackOverflow Developer Survey Results 2016 (it was "most loved" in 2015). Given the target environments for Swift (macOS, iOS, watchOS and tvOS) and without even considering the language's virtues, it's not surprising that it's one of the most commonly used programming languages.

Go

On the Tiobe Index, "Google's Go is TIOBE's programming language of 2016." In other words, Tiobe found that Go was "the programming language that has gained the most popularity" in 2016. Go ended the year in the 13th overall popularity spot with familiar languages such as Java, C/C++/C#, Python, JavaScript, Perl, and PHP still ahead of it overall. The Tiobe page provides explanations for Go's rise in popularity: "The main drivers behind Go's success are its ease of learning and pragmatic nature. It is ... about hands-on experience [and] Go has built-in support for concurrency programming. More and more customers of our company TIOBE are adopting Go in an industrial setting." Of course, everyone has their own definition of how to measure language popularity. Tiobe's site explains their methodology at a high level.

2016 included the seventh anniversary of the "preliminary sketch of Go" being open sourced and the Seven years of Go blog post summarizes some of Go's biggest happenings in 2016. One of these developments was the release of Go 1.7.

Conclusion

Much has happened in the world of software development in 2016. In this post, I have looked at a small subset of these developments. I may add a few more honorable mention items in the next week or two, but wanted to get this published while still in 2016.

Friday, December 9, 2016

Compare Exactly in Java Equals Methods

As I've worked with legacy Java code over the years, I've run into subtle logic and performance issues that could be traced back to improperly overridden Object.equals(Object) methods. Although the concept behind the "equals" method is seemingly simple, Josh Bloch points out in Effective Java that "Overriding the equals method seems simple, but there are many ways to get it wrong, and the consequences can be dire. The easiest way to avoid problems is not to override the equals method, in which case each instance is equal only to itself." In this post, I look at one of "the many ways" to get an equals(Object) wrong: failing to compare exactly the same characteristics of the two objects being evaluated for equality.

The next code listing is for class MismatchedFieldAccessor. This class's equals(Object) method is flawed because it compares the class's direct attribute someString to the value retrieved from the other object's getSomeString(). In most Java classes, comparing a class's field to its accessor/get method will work properly because the accessor/get method simply returns the associated field. In this example class, however, the accessor/get method does more than simply returning the field and that makes the comparison of the field to the get/accessor method in the equals(Object) method inconsistent. (Note that the idea of having a "get" method do this type of thing is not being recommended here, but merely exists as a simple-to-understand example.)

package dustin.examples.brokenequals;

import java.util.Objects;

/**
 * Demonstrate problem with mismatched field/accessor in
 * overridden equals(Object) implementation.
 */
public final class MismatchedFieldAccessor
{
   private final String someString;

   public MismatchedFieldAccessor(final String newString)
   {
      someString = newString;
   }

   public String getSomeString()
   {
      return someString != null ? someString : "";
   }

   @Override
   public boolean equals(final Object other)
   {
      if (this == other)
      {
         return true;
      }
      if (other == null || getClass() != other.getClass())
      {
         return false;
      }

      final MismatchedFieldAccessor that = (MismatchedFieldAccessor) other;

      return Objects.equals(this.someString, that.getSomeString());
   }

   @Override
   public int hashCode()
   {
      return someString != null ? someString.hashCode() : 0;
   }
}

The above class will fail if tested with an appropriate unit test. The two unit tests listed in the next code listing point out issues with the class's equals method.

public void testEqualsOnConstructedWithNull()
{
   final MismatchedFieldAccessor accessor = new MismatchedFieldAccessor(null);
   Assert.assertEquals(null, accessor.getSomeString());
}

@Test
public void testEqualsWithEqualsVerifier()
{
   EqualsVerifier.forClass(MismatchedFieldAccessor.class).verify();
}

The first unit test shown above fails with this message:

java.lang.AssertionError: 
Expected :null
Actual   :

The second unit test makes use of the handy EqualsVerifier library to identify an issue with this equals(Object) implementation (emphasis added):

java.lang.AssertionError: Reflexivity: object does not equal an identical copy of itself:
  dustin.examples.brokenequals.MismatchedFieldAccessor@0
If this is intentional, consider suppressing Warning.IDENTICAL_COPY
For more information, go to: http://www.jqno.nl/equalsverifier/errormessages

 at nl.jqno.equalsverifier.EqualsVerifier.handleError(EqualsVerifier.java:381)
 at nl.jqno.equalsverifier.EqualsVerifier.verify(EqualsVerifier.java:367)
 at dustin.examples.brokenequals.MismatchedFieldAccessorTest.testEqualsWithEqualsVerifier(MismatchedFieldAccessorTest.java:36)

This post has covered one of the many ways in which an equals method can go bad if not carefully implemented, reviewed, and tested. Fortunately, the fix for this particular problem is easy: always compare the exact same field or same method's returned object of the two instances being compared for equality. In the example used in this post, comparing the two "someString" fields directly would have made the "equals" method work properly.

Wednesday, November 23, 2016

Bash on Windows 10

Because I work with Linux and Windows based machines for development, I often find myself wishing that I had some of the handy command-line Linux tools available in my Windows environments. Cygwin, PowerShell, and custom Groovy scripts written to emulate Linux tools have helped, but I was pleasantly surprised to recently learn that Bash on Ubuntu on Windows 10 is available. In this post, I briefly summarize some of the steps to make bash available on Windows. More detailed instructions with helpful screen snapshots can be found in the Installation Guide.

The Windows Subsystem for Linux (WSL) is described in the Frequently Asked Questions page as "a new Windows 10 feature that enables you to run native Linux command-line tools directly on Windows, alongside your traditional Windows desktop and modern store apps." This same FAQ page states that enabling WSL "downloads a genuine Ubuntu user-mode image, created by Canonical."

The following are the high-level steps for getting Windows 10 ready to use WSL bash.

  1. Verify that Windows 10 installation is 64-bit "system type" and has an "OS Build" of at least 14393.0.
  2. Turn on "Developer Mode"
  3. Enable Windows Subsystem for Linux (WSL) ["Turn Windows Features On or Off" GUI]
  4. Enable Windows Subsystem for Linux (WSL) [PowerShell Command-line]
  5. Restart Computer as Directed
  6. Run bash from Command Prompt (Downloads Canonical's Ubuntu on Windows)
  7. Create a Unix User

Once the steps described above have been executed, bash can be easily used in the Windows 10 environment. A few basic commands are shown in the next screen snapshot. It shows running bash in the Command Prompt and running a few common Linux commands while in that bash shell.

As a developer who uses both Windows and Linux, having bash available in Windows 10 is a welcome addition.

Related Online Resources

Monday, November 21, 2016

Inheriting Javadoc Method Comments

Although the JDK Tools and Utilities pages for the javadoc tool describe the rules of Javadoc method comment reuse by implementing and inheriting methods, it is easy to unnecessarily explicitly describe comment inheritance with {@inheritDoc} when it's not really needed because the same comments would be implicitly inherited. The Java 8 javadoc tool page describes the rules of inherited method Javadoc comments under the section "Method Common Inheritance" and the Java 7 javadoc tool page similarly describes these rules under the section "Automatic Copying of Method Comments." This post uses simple code examples to illustrate some of the key rules of Javadoc method comment inheritance.

The following interfaces and classes are contrived examples that will be used in this post to illustrate inheritance of Javadoc comments on methods. Some inherited/implementing methods include their own Javadoc comments that override parent's/interface's methods comments fully or partially and other simply reuse the parent's/interface's methods' documentation.

Herbivorous Interface

package dustin.examples.inheritance;

/**
 * Marks animals that eat plants.
 */
public interface Herbivorous
{
   /**
    * Eat the provided plant.
    *
    * @param plantToBeEaten Plant that will be eaten.
    */
   void eat(Plant plantToBeEaten);
}

Carnivorous Interface

package dustin.examples.inheritance;

/**
 * Marks an Animal that eats other animals.
 */
public interface Carnivorous
{
   /**
    * Eat the provided animal.
    *
    * @param animalBeingEaten Animal that will be eaten.
    */
   void eat(Animal animalBeingEaten);
}

Omnivorous Interface

package dustin.examples.inheritance;

/**
 * Eats plants and animals.
 */
public interface Omnivorous extends Carnivorous, Herbivorous
{
}

Viviparous Interface

package dustin.examples.inheritance;

/**
 * Mammals that give birth to young that develop within
 * the mother's body.
 */
public interface Viviparous
{
   /**
    * Give birth to indicated number of offspring.
    *
    * @param numberOfOffspring Number of offspring being born.
    */
   void giveBirth(int numberOfOffspring);
}

Animal Class

package dustin.examples.inheritance;

/**
 * Animal.
 */
public abstract class Animal
{
   /**
    * Breathe.
    */
   public void breathe()
   {
   }

   /**
    * Communicate verbally.
    */
   public abstract void verballyCommunicate();
}

Mammal Class

package dustin.examples.inheritance;

/**
 * Mammal.
 */
public abstract class Mammal extends Animal
{
}

MammalWithHair Class

package dustin.examples.inheritance;

import java.awt.*;

/**
 * Mammal with hair (most mammals other than dolphins and whales).
 */
public abstract class MammalWithHair extends Mammal
{
   /** Provide mammal's hair color. */
   public abstract Color getHairColor();
}

Dog Class

package dustin.examples.inheritance;

import java.awt.Color;

import static java.lang.System.out;

/**
 * Canine and man's best friend.
 */
public class Dog extends MammalWithHair implements Omnivorous, Viviparous
{
   private final Color hairColor = null;

   /**
    * {@inheritDoc}
    * @param otherAnimal Tasty treat.
    */
   @Override
   public void eat(final Animal otherAnimal)
   {
   }

   /**
    * {@inheritDoc}
    * @param plantToBeEaten Plant that this dog will eat.
    */
   @Override
   public void eat(final Plant plantToBeEaten)
   {
   }

   /**
    * {@inheritDoc}
    * Bark.
    */
   public void verballyCommunicate()
   {
      out.println("Woof!");
   }

   /**
    * {@inheritDoc}
    * @param numberPuppies Number of puppies being born.
    */
   @Override
   public void giveBirth(final int numberPuppies)
   {
   }

   /**
    * Provide the color of the dog's hair.
    *
    * @return Color of the dog's fur.
    */
   @Override
   public Color getHairColor()
   {
      return hairColor;
   }
}

Cat Class

package dustin.examples.inheritance;

import java.awt.Color;

import static java.lang.System.out;

/**
 * Feline.
 */
public class Cat extends MammalWithHair implements Carnivorous, Viviparous
{
   private final Color hairColor = null;

   /**
    * {@inheritDoc}
    */
   @Override
   public void eat(final Animal otherAnimal)
   {
   }

   @Override
   public void verballyCommunicate()
   {
      out.println("Meow");
   }

   @Override
   public void giveBirth(int numberKittens)
   {
   }

   @Override
   public Color getHairColor()
   {
      return hairColor;
   }
}

Horse Class

package dustin.examples.inheritance;

import java.awt.Color;

import static java.lang.System.out;

/**
 * Equine.
 */
public class Horse extends MammalWithHair implements Herbivorous, Viviparous
{
   private final Color hairColor = null;

   /**
    * @param plant Plant to be eaten by this horse.
    */
   @Override
   public void eat(final Plant plant)
   {
   }

   /**
    *
    */
   @Override
   public void verballyCommunicate()
   {
      out.println("Neigh");
   }

   /**
    * @param numberColts Number of colts to be born to horse.
    */
   @Override
   public void giveBirth(int numberColts)
   {
   }

   @Override
   public Color getHairColor()
   {
      return hairColor;
   }
}

The next screen snapshot shows the contents of the package that includes the interfaces and classes whose code listings are shown above (not all the classes and interfaces in the package had their code listings shown).

The three classes of most interest here from methods' Javadoc perspective are the classes Dog, Cat, and Horse because they implement several interfaces and extend MamalWithHair, which extends Mammal, which extends Animal.

The next screen snapshot is of the Javadoc for the Animal class rendered in a web browser.

The Animal class doesn't inherit any methods from a superclass and doesn't implement any methods from an interface and is not very interesting for this blog post's topic. However, other classes shown here extend this class and so it is interesting to see how its method comments affect the inheriting classes' methods' descriptions.

The next two screen snapshots are of the Javadoc for the Mammal and MammalWithHair classes as rendered in a web browser. There are no Javadoc comments on any significance on Mammal, but there is one method comment for a new method introduced by MammalWithHair.

The next three screen snapshots are of subsets of Javadoc documentation in a web browser for the interfaces Herbivorous, Carnivorous, and Omnivorous. These interfaces provide documentation for methods that will be inherited by classes that implement these methods.

With the generated Javadoc methods documentation for the parent classes and the interfaces shown, it's now time to look at the generated documentation for the methods of the classes extending those classes and implementing those interfaces.

The methods in the Dog class shown earlier generally used {@inheritDoc} in conjunction with additional text. The results of inheriting method Javadoc comments from extended classes and implemented interfaces combined with additional test provided in Dog's comments are shown in the next screen snapshots.

The last set of screen snapshots demonstrates that the Dog class's documentation mixes the documentation of its "parents" with its own specific documentation. This is not surprising. The Dog class's methods generally explicitly inherited Javadoc documentation from the parents (base classes and interfaces), but the Cat class mostly has no Javadoc comments on its methods, except for the eat method, which simply uses {@inheritDoc}. The generated web browser output from this class is shown in the next screen snapshots.

The methods in Cat that had no Javadoc comments applied show up in the generated web browser documentation with documentation inherited from their base classes or interfaces and the documentation on these methods includes the phrase "Description copied from class:" or "Description copied from interface:" as appropriate. The one Cat method that does explicitly include the documentation tag {@inheritDoc} does copy the parent's method's documentation, but does not include the "Description copied from" message.

The Horse class's methods are generally not documented at all and so their generated documentation includes the message "Description copied from...". The eat() and giveBirth() methods of the Horse class override the @param portion and so the parameter documentation for these two methods in the generated web browser documentation (shown in the next set of screen snapshots) is specific to Horse.

From the above code listings and screen snapshots of generated documentation from that code, some observations can be made regarding the inheritance of methods' Javadoc comments by extending and implementing classes. These observations are also described in the javadoc tool documentation:

  • Javadoc comments are inherited from parent class's methods and from implemented interface methods either implicitly when no text is specified (no Javadoc at all or empty Javadoc /** */).
    • javadoc documentation: "The javadoc command allows method comment inheritance in classes and interfaces to fill in missing text or to explicitly inherit method comments."
  • Use {@inheritDoc} explicitly states that comments should be inherited.
    • javadoc documentation: "Insert the {@inheritDoc} inline tag in a method main description or @return, @param, or @throws tag comment. The corresponding inherited main description or tag comment is copied into that spot."
  • Implicit and explicit inheritance of method documentation can be achieved in combination by using {@inheritDoc} tags in different locations within the method comment.

Given the above observations and given the advertised "Method Comments Algorithm", a good rule of thumb for writing Javadoc from the perspective of the HTML generated from the Javadoc is to define general comments at as high a level as possible and allow automatic inheritance of the extended classes's and implemented interfaces' methods' Javadoc documentation to take place, adding or overriding only portions of a method's Javadoc text that are necessary to clarify or enhance the description for a lower-level method. This is better than copying and pasting the same comment on all methods in an inheritance or implementation hierarchy and needing to then keep them all updated together.

This post has focused on the web browser presentation of generated Javadoc methods' documentation. Fortunately, the most commonly used Java IDEs (NetBeans [CTRL+hover], IntelliJ IDEA [CTRL+Q / Settings], Eclipse [F2 / hover / Javadoc View], and JDeveloper [CTRL-D]) support presentation of Javadoc that generally follows the same rules of method documentation inheritance. This means that Java developers can often write less documentation and almost entirely avoid repeated documentation in inheritance and implementation hierarchies.