Craft, cooperative gaming and lessons from lean manufacturing

Alistair Cockburn, in his thought-provoking publications titled "From Agile Development to the New Software Engineering", shows how software engineering is built on the following three legs and what to make of it. A few excerpts:

• Craft -  "Being in a craft profession brings with it certain expectations, such as lifelong learning, or “deepening” the proficiency in one’s craft. Programmers shouldn’t think that just because they once learned to program a computer, their programming skills are still sufficient. As craft professionals, programmers, project managers and others should always be looking to learn new tools, and techniques, and get better in the ones they have. [Cockburn]"
• Cooperative gaming - "Every game and every situation are potentially different – there is no formula for winning the game. Different, even opposite strategies may be needed at any instant; The quality of a move in the game is not absolute, but rather, is only relative to how it improves the position of the team for its next move; The quality of community and communication among the team members matter enormously – they often make the difference between success and failure. [Cockburn]"
• Lessons from lean manufacturing - "... people hand other people decisions; people wait on each other for decisions; and some people have a bigger backlog of decisions than they can handle at the moment. Examining the dependency network of decisions in play in an organization, we see that this network is very similar to the dependency network of parts in a manufacturing plant. Quite surprisingly, the same mathematics applies to the situation, and many of the same strategies apply: just-in-time, pull, continuous flow, and so on. [Cockburn]"

Software Engineering Method and Theory (SEMAT)

Anyone who has worked in the software industry for a few years, in all likelihood, wouldn't find it hard to acknowledge that the state of practice in software development has a long way to go.
 
Visit Software Engineering Method and Theory (SEMAT) for a community working on developing a framework for improving the state of practice in software. Its heartening to see a list of great signatories on this initiative.
 
The SEMAT call for action states:

Software engineering is gravely hampered today by immature practices. Specific problems include:  

• The prevalence of fads more typical of fashion industry than of an engineering discipline.
• The lack of a sound, widely accepted theoretical basis.
• The huge number of methods and method variants, with differences little understood and artificially magnified.
• The lack of credible experimental evaluation and validation.
• The split between industry practice and academic research.

Visit here for SEMAT publications and references.

Data view as an architectural view

For a nice roundup of data view as an architectural view, refer to Paulo Merson's tech note on the SEI site.

How we overvalue stuff

Paul Graham on "Stuff". He says "Stuff has gotten a lot cheaper, but our attitudes toward it haven't changed correspondingly. We overvalue stuff." So true.

Enlivening discussion around static and dynamic types

For a hilarious account of "Static-Typing Fundamentalist, the bearded and grizzled veteran of the Static/Dynamic Code Wars [Ted Neward] ", read Ted Neward's blog post on the comments posted on Scott Hanselman's intro to the C# dynamic types (don't miss the comments on Scott's post, either).

CWE's list of top 25 most dangerous programming errors

Common Weakness Enumeration (CWE)'s study describing the top 25 programming errors that can open up security holes in your applications is here.

Examples of Interoperability Issues in SOAP-based Web services

1- Mismatch in platform native data types

Data type compatibility is among the most important issue in Web services (SOAP based) interoperability. SOAP implementations convert XML schema types to language native data types during serialization and vice versa during deserialization. Different SOAP runtimes and underlying platforms/languages at sender and receiver end can therefore cause interoperability issues.

a) No direct native type maping. Some schema types such as xsd:unsignedshort, xsd:unsignedint xsd:unsignedlong do not always have a direct native type mappings on all languages. For example, in Java, unsigned types are not defined.

b) Difference in precisions. Different programming languages and platorms might support different precisions for XML schema defined floating point and decimal numbers such as xsd:decimal, xsd:double or xsd:float, leading to loss of precision during translation between xsd types to native types.
A good example of this issue is that of supporting Bigdecimal. XML Schema specification for decimal data types allows an arbitrary precision, upto 40 digits. Although Java BigDecimal also is defined as arbitrary-precision signed decimal numbers, it has an upper limit based on the underlying platform - Windows, Solaris, Linux, etc. The .NET implementation supports upto 29 digits.

Similarly, platforms use different precisions when interpreting native xsd:dateTime types. As per XML Schema specification, any number of digits after seconds may be coded into a dateTime data type. The Java platform supports a precision of upto three digits for the millisecond while .NET supports seven digits for the millisecond (can therefore support nanoseconds).

c) Null values in SOAP messages. Difference in how schema data types are mapped to native platform data types can cause issue in handling null values.
For example, lets say, a Java based Web service returns a date/time using java.util.Date or java.util.Calendar and a .NET consumer consumes the service. If the service returns a null, the .NET consumer fails, because in .NET System.DateTime is considered a value type (i.e., they reside in the stack) and value types cannot be a null. This is unlike Java where the two are classed as reference types (stored in the heap) and therefore can be null.

d) Processing byte arrays. SOAP allows exchanging data in the form of byte arrays (e.g., base64 encoded and placed on the XML document). Implementations may return inconsistent values due to a difference in native data types or other problems.

e) Processing nil and minOccurs='0'. These get handled differently in different platforms. For example, Java tools tend to serialize nulls explicitly as xsi:nil='true' unless told otherwise.

f) Weakly typed data structures: Some programming languages support Collection types such as hashmaps, vectors, sets, etc. Objects of these collection types, contains objects of other types. The receiving side may have trouble understanding SOAP messages containing such weakly-typed elements.
For example, an ArrayList may be taken as "anyType" in the XML schema, making it hard for the receiving side to map it to appropriate type.

2- Platform Idiosyncracies

a) Different interpretations of namespace declarations. Lets say you have two .NET services defined by unique URLs of the form: http://domain/branchname in the namespace declarations. The first service contains the URL: http://www.somecompany.com/ordering in its namespace declaration and the other contains the URL: http://www.somecompany.com/provisioning. Although both have unique URLs in the namespace declarations, a Java consumer consuming both services will face a naming conflict. This is because, the package name in generated stub code in Java is based on the domain name of the namespace, unlike in .NET where both domain name and the branch name are used to construct the namespace of the generated code.

b) Difference in platform generated proxies. Service proxies/stubs generated using one implemention do not typically work with other implementations. Moreover, code written against proxies generated using one version of the implementation may not work with another version of the implementation. For example, Code written against Axis 1.1 generated proxy may not work when the proxy is regenerated using Axis 1.2 (for example due to evolving xsd to Java type mappings).

c) Differences in expectations of type information. SOAP 1.1 defines a way to type each value explicitly using the xsi:type attribute. This is optional and typically not required if the type information is being expressed through the use of schema or any form of metadata exchange.
There were implementations that require explicit type information such as an earlier version of Apache SOAP. Other implementations such as .NET SOAP implementations didn't support it at all. This caused significant interoperability issues, since the implementation requiring type information would not be able to process a message from an implementation that doesn't include explicit type information.

3- SOAP and WSDL specific issues

a) Usage of mustUnderstand attribute. SOAP 1.1 defines a mustUnderstand attribute that can be specified for a SOAP header. The intent of this attribute is to ensure that if a SOAP processor/implementation doesn't understand it, it must fault. In other words, the recipient is required to understand and make use of the information supplied by the header. This helps in situations where a recipient can't perform its job unless it knows what to do with data provided by this header element.

SOAP 1.1 defines values for this attribute as 0 (false) or 1(true). The issues related to this attribute are:

• Some toolkits have allowed the values "false" and "true". Latest SOAP versions may also be requiring implementations to support these.
• Some SOAP runtimes do not support this attribute and rather offer work-arounds.

b) Out-of-sync SOAPAction headers. SOAP defines a "SOAPAction" request header. When bound to HTTP, SOAP mandates using this header, i.e., an HTTP client must issue this header field when issuing a SOAP HTTP request. According to http://www.w3.org/TR/2000/NOTE-SOAP-20000508/#_Toc478383528, this can be used to indicate the intent of the SOAP HTTP request. Legal values of this header's value are:

• null or no value, meaning there is no indication of the intent of the message, e.g., SOAPAction:
• Empty value, meaning the intent of the message is provided by the HTTP request URI, e.g., SOAPAction:""
• An arbitrary URI, such as SOAPAction: "http://mydomain.com/whatever" or SOAPAction: "whatever.sdl". Implementations have also allowed unquoted URIs such as SOAPAction:http://mydomain/whatever.

The original intent of this header was to give hints about the content of the message to devices at lower levels (e.g., proxies, HTTP intermediaries, firewalls for filtering), without requiring the recipient to parse the XML. This allows implementations to identify, route or dispatch requests without having to parse the input request.

Some implementations also use it for application-level dispatch (e.g., routing the message to appropriate code).

Issues related to this header are:

• Use of SOAPAction for transports other than HTTP, is not specified by SOAP 1.1., although some SOAP implementations do provide equivalent functionality for other transport protocols, leading to interoperability issues, since not all implementations implement or implement the same way.
• SOAP implementations may interpret the "intent of the SOAP message" in different ways. This could be interpreted as the intended target for the message, the name of the target service, application level dispatch, and so forth. Therefore, a unless prior negotiation occurs between the sender and the receiver, full interoperability between implementations is hard to achieve.
• Some SOAP implementations require that the values be quoted. However, some implementations do not quote their SOAPAction HTTP header values; this causes problems with implementations that assume SOAPAction values.
• Also, if a receiver requires null-value SOAPAction, some HTTP client APIs have difficulty setting a null HTTP header value.
• Some implementations are unable to parse the the Unicode byte order mark sent by other implementations.
• Some SOAP implementations such as Apache SOAP do not support dispatching using the SOAPAction header. Instead they route based on the namespace URI of the first child element contained in the SOAP:Body element.

c) Identifying SOAP fault: Some implementations use only the HTTP status code to determine the presence of SOAP Fault. However, there are cases where the web infrastructure changes the HTTP status code.

d) WSDL Style and encoding. The WSDL "style" refers to the way WSDL describes the content of a SOAP message. It has two legal values - Document and RPC.

WSDL also has a "use" attribute (encoded/literal) that controls how web services messages are serialized (formatted according to a standard format).

• SOAP 1.1 defines a SOAP encoding scheme (also refererred to as Section 5 encoding) that uses a set a rules based on the XML schema data types to encode programming language data types to XML and vice versa. However, the message doesn't conform to a particular schema. SOAP encoding is particularly useful while passing cyclic object graphs. For e.g., of one has an object graph that references a particular element many times, one would specify the element once and reference it at other places (similar to how one would do in a programming language such as Java). In literal encoding, on the other hand, one would have to repeat the element each time it is referenced. However, on the flip side, one cannot validate the message with an XML schema or transform it with XSLT.
• SOAP also supports literal encoding, which means the application data such as objects, primitive types and methods are serialied to XML according to a particular schema, and therefore one can validate the message with an XML schema or transorm it using XSLT.

Key interoperability issues related to the value of the styles (RPC/Document) and uses (encoded/literal) attributes in WSDL are:

• SOAP has no default encoding. Also, encoding can be defined through the encodingStyle attribute that identifies serialization rules. Some SOAP implementations default to SOAP encoding, while others default to using literal encoding. So, one side must be able to specify the encoding explicitly rather than use its default.
• Based on the WSDL style and use attributes, these are these possible combinations, that implementations must agree to (or worked around) before two sides can interoperate: RPC/Encoded, RPC/Literal, Document/Encoded (almost never used) and Document/Literal (and Document/Literal Wrapped).

e) Mismatch using the "actor" attribute. A SOAP message sometimes passes through one or more intermediaries (such as a SOAP proxy service) before being processed. Therefore, SOAP headers may contain information intended solely for an intermediary, as well as that for the ultimate destination. If an "actor" attribute doesn't appear on the header element, it is assumed to be intended for the ultimate destination. SOAP requires that a header element intended for an intermediary is not passed on to the next SOAP processor in the path.

An actor is of the type anyURI and can have multiple URI identifiers. The ultimate destination can be designated either explicitly (targeting the ultimate destination using a URI identifier) or by the absence of the actor attribute, as described in the last paragraph. This leads to confusion and interoperability issues among different SOAP implementations.

Also, certain SOAP implementations put additional constraints on SOAP headers containing "actor" attributes. For example, .NET WSE2.0 server looks for the WS-Addressing header when the actor attribute is configured on the client, while the clients may not be requiring that, since its not required by SOAP specification.

For additional interoperability issues refer to http://www.xmethods.net/soapbuilders/interop.html.

Get started with Google App Engine

Google App Engine allows you to deploy apps in Google's production infrastructure (for free upto a limit and then using pay-as-you-go model). Key advantages include:

• You don't need to worry about handling scaling (delivery on a global scale and low latency), so you can concentrate on app's features and innovations
• OpenSocial, Android devices, Facebook, etc. opens up your app to 100s of millions of target users
• Typical to Google solutions, costs are low and fair

Get started with App Engine with Rick Hightower's three-part article series:
Google App Engine for Java: Part 1: Rev it up!
Google App Engine for Java: Part 2: Building the killer app
Google App Engine for Java: Part 3: Persistence and relationships
Ofcourse, you could also start here: http://code.google.com/appengine/

Management 2.0

One of my favourites books on Management is "The Future of Management" by Gary Hamel and Bill Breen. A few excerpts from the book are as follows:

(To follow Gary via his blog, subscribe to his blog here.)

"Ask a group of your colleagues to describe the distinguishing characteristics of your company, and few are likely to mention adaptability and inventiveness. Yet if you ask them to make a list of the traits that differentiate human beings from other species, resilience and creativity will be near the top of the list. We see evidence of these qualities every day—in ourselves and in those around us. All of us know folks who’ve switched careers in search of new challenges or a more balanced life. We know people who’ve changed their consumption habits for the sake of the planet. We have friends and relatives who’ve undergone a spiritual transformation, or risen to the demands of parenthood, or overcome tragedy. Every day we meet people who write blogs, experiment with new recipes, mix up dance tunes, or customize their cars. As human beings, we are amazingly adaptable and creative, yet most of us work for companies that are not. In other words, we work for organizations that aren’t very human.
... The culprit? Management principles and processes that foster discipline, punctuality, economy, rationality, and order, yet place little value on artistry, nonconformity, originality, audacity, and élan. To put it simply, most companies are only fractionally human because they make room for only a fraction of the qualities and capabilities that make us human. Billions of people show up for work every day, but way too many of them are sleepwalking. The result: organizations that systematically underperform their potential.
.... The challenge: to reinvent our management systems so they inspire human beings to bring all of their capabilities to work every day."

"... Hierarchies are very good at aggregating effort, at coordinating the activities of many people with widely varying roles. But they’re not very good at mobilizing effort, at inspiring people to go above and beyond. When it comes to mobilizing human capability, communities outperform bureaucracies."

"... also hear virtually nothing that suggests the participants have hearts. Beauty. Truth. Love. Service. Wisdom. Justice. Freedom. Compassion. These are the moral imperatives that have aroused human beings to extraordinary accomplishment down through the ages. It is sad, then, that the vernacular of management has so little room for these virtues. Put simply, you are unlikely to get bighearted contributions from your employees unless they feel they are working toward some goal that encompasses bighearted ideals."