Wednesday, October 26, 2016

Many to Many Entity Framework 6.0 Relationships Without Extra Nesting

So you've got a many to many (m2m) relationship. And you'd like the Entity Framework to return both ends of this relationship using SQL that isn't overly nested/CASE-d/UnionAll-d.

Let's take the canonical m2m relationship of Authors to Books. An Author can write many Books and a Book can be written by many Authors.

An Author

Many Books
Depending on how well the Entity Framework can figure out the multiplicity of each end you may end up with SQL that's very inefficient.

A human would query this with LEFT OUTER JOINs.

But given the following C# code (LINQ to Entities, query syntax):


from a in AUTHORS
select new { a, a.BOOKS }

The generated SQL has too much nesting and aliasing:

SELECT 
"Project1"."C1" AS "C1", 
"Project1"."AUTHOR_ID" AS "AUTHOR_ID", 
"Project1"."FNAME" AS "FNAME", 
"Project1"."LNAME" AS "LNAME", 
"Project1"."C2" AS "C2", 
"Project1"."BOOK_ID" AS "BOOK_ID", 
"Project1"."NAME" AS "NAME", 
"Project1"."DESCRIPTION" AS "DESCRIPTION"
FROM ( SELECT 
 "Extent1"."AUTHOR_ID" AS "AUTHOR_ID", 
 "Extent1"."FNAME" AS "FNAME", 
 "Extent1"."LNAME" AS "LNAME", 
 1 AS "C1", 
 "Join1"."BOOK_ID1" AS "BOOK_ID", 
 "Join1"."NAME" AS "NAME", 
 "Join1"."DESCRIPTION" AS "DESCRIPTION", 
 CASE WHEN ("Join1"."BOOK_ID2" IS NULL) THEN NULL ELSE 1 END AS "C2"
 FROM  "SCOTT"."AUTHORS" "Extent1"
 LEFT OUTER JOIN  (SELECT "Extent2"."BOOK_ID" AS "BOOK_ID2", "Extent2"."AUTHOR_ID" AS "AUTHOR_ID", "Extent3"."BOOK_ID" AS "BOOK_ID1", "Extent3"."NAME" AS "NAME", "Extent3"."DESCRIPTION" AS "DESCRIPTION"
  FROM  "SCOTT"."AUTHOR_BOOKS" "Extent2"
  INNER JOIN "SCOTT"."BOOKS" "Extent3" ON "Extent3"."BOOK_ID" = "Extent2"."BOOK_ID" ) "Join1" ON "Extent1"."AUTHOR_ID" = "Join1"."AUTHOR_ID"
)  "Project1"
ORDER BY "Project1"."AUTHOR_ID" ASC, "Project1"."C2" ASC

This holds true even if eager loading (.Include("BOOKS")) is used. It would be nice to use a LEFT OUTER JOIN here without the extra nesting.

Modified C# to eliminate nesting:

from a in AUTHORS
from b in a.BOOKS.DefaultIfEmpty()
select new { a, b }

Eliminates the extra level of nesting in the resulting SQL:

SELECT 
1 AS "C1", 
"Extent1"."AUTHOR_ID" AS "AUTHOR_ID", 
"Extent1"."FNAME" AS "FNAME", 
"Extent1"."LNAME" AS "LNAME", 
"Join1"."BOOK_ID1" AS "BOOK_ID", 
"Join1"."NAME" AS "NAME", 
"Join1"."DESCRIPTION" AS "DESCRIPTION"
FROM  "SCOTT"."AUTHORS" "Extent1"
LEFT OUTER JOIN  (SELECT "Extent2"."BOOK_ID" AS "BOOK_ID2", "Extent2"."AUTHOR_ID" AS "AUTHOR_ID", "Extent3"."BOOK_ID" AS "BOOK_ID1", "Extent3"."NAME" AS "NAME", "Extent3"."DESCRIPTION" AS "DESCRIPTION"
 FROM  "SCOTT"."AUTHOR_BOOKS" "Extent2"
 INNER JOIN "SCOTT"."BOOKS" "Extent3" ON "Extent3"."BOOK_ID" = "Extent2"."BOOK_ID" ) "Join1" ON "Extent1"."AUTHOR_ID" = "Join1"."AUTHOR_ID"

This example is based on using Entity Framework 6.0 with Oracle's ODP.NET Managed Entity Framework driver. That driver, in turn, depends on the ODP.NET Managed Framework driver (strangely enough, at least at first glance, these are not the same thing).

Wednesday, September 14, 2016

Parser-Friendly Angular 1.3.* Formatting

Angular 1.3.* controllers were initially formatted as follows:

1
2
3
angular.module('path.to.moduleName').controller(['$scope', function($scope) {
  //...
}];

This style has its advantages, particularly while learning angular, in that it reinforces the various concepts (modules, app, dependencies) that are important in the design of the framework.

But unfortunately this style tends to stymie many JavaScript parsers. One widely used library for parsing languages is called Ctags. Many editors rely on this library for parsing. And given a sufficiently long controller using the aforementioned format will easily break the parser.

With a small change in formating we can get the benefit of parsers (e.g., Jump To Definition) while working in JavaScript. With such a free-form language I'll take all of the tool help I can get.


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
var myCtrlFunc = function() {
    // ... code goes here
    function fun1() {
        //...
    };
    
    $scope.fun2 = function() {
        //...
    };
};

angular.module('path.to.module').controller('myCtrl', [myCtrlFunc]);

Wednesday, September 7, 2016

What Is Windows Up To Right Now?

Years ago one relied on the hard drive light for reassurance during some particularly unresponsive episode on the PC. The reassuring hum of those spindles spinning formed a kind of non-deterministic progress indicator.

Spin drives gave way to Solid State Drives (SSDs) which afford no reassuring hum. For a while the activity light remained but nowadays it seems that even that has entered the dustbin of PC history. Sort of loses its usefulness if it's always on...

So, like pretty much every commercial concern, I turned to software. Windows has shipped with Resource Monitor (resmon) for several versions. It augments Task Manager's Performance tab (so much so that a button to open it was added to that tab in windows 7).

For answering the general question "What is Windows doing right now?" I maximize the Overview tab on the right monitor. So far the Disk activity grid seems to yield the greatest insight into what's going on at any given instant.

For example, just watching the Disk activity grid while loading a web app revealed excessive logging settings leftover from previous debugging efforts. You can directly see iisexpress.exe writing to the trace logs.

Here are a few tips for getting the most out of resmon's Overview tab:

  1. Increase the height of the Disk grid. It should be, at least, double the height of grids.
  2. Widen the Disk grid's image and file columns. The file column should take up most of the space to account for very long pathnames.
  3. Sort the Disk grid's by Total (B/sec) column in descending order. This brings the files with the most disk activity to the top.
  4. Sort the CPU grid by Average CPU descending.
  5. Sort the Network grid by Total (B/sec) descending.
  6. Sort the Memory grid by Working Set (KB)

Wednesday, August 31, 2016

Single Page Applications And The Impending Concurrency Explosion

Ok explosion might be a little dramatic but bear with me for a moment.

Going from a Multi Page Application (MPA, did I just coin a retronym?) to a Single Page Application (SPA) is like going from a single synchronization lock (coarse-grained locking) to multiple synchronization locks (fine-grained locking).

Anyone who has made that journey knows that it is fraught with peril. Even for a solo developer it can be difficult to implement without error. For a team of developers, with varying level of application domain familiarity and developer expertise, it is far more daunting.


How is going from an MPA to an SPA like this concurrency problem?

With an MPA the full page submit is analogous to acquiring a single lock. The state of the app is updated in one fell swoop.

With an SPA the same amount of state will be distributed across separate async requests that fire, largely, in response to user interaction. Since the user drives the requests any ordering requirements must be enforced by the developer(s).

Each of these requests is like acquiring one of many locks (e.g., read-only consumers only need the read lock, which is shareable, whereas read-write consumers need both the read and write lock (not shareable)). On the user interface, independently queryable data can update different UI widgets in parallel*. This increases the perceived interactivity of the web app but comes at the cost of increased complexity.

It can be difficult to coordinate multiple lock scenarios even when all of the developers working on the project are co-located. Part of this difficulty stems from the difficulty modeling concurrency - there are many notations available and none used by everyone.

Throw in developers distributed geographically, varying levels of expertise, even varying languages and the coordination required to prevent lock-related errors becomes quite substantial.

The theory of it is sound but in practice there are often subtle data dependencies that don't reveal themselves when all data is fetched and the page is rendered in one fell swoop. That is, the dependencies don't show up until teasing apart the data needed for the various dynamic portions of the page. In other words, the data access patterns of an SPA are going to be different than the data access patterns of an MPA.

Wednesday, August 24, 2016

Angular And Filtering Exactly

Angular ships with a set of filters for common output transformations (e.g., formatting a date, formatting currency, etc...).

The filter provider also exposes the 'filter' filter. This filter provides a way to query an array of objects based on property values. It's not as sophisticated as LINQ but it's better than nothing.

By default, the filter 'filter' does a substring match instead of an exact match.

For example, the following code snippet:


$filter('filter')([{name: 'one', male: false}, {name: 'two', male: 'trueisms'}, {name: 'three', male: true}], {male: true})

returns a list with 2 objects (male = 'trueisms' and male = true) instead of 1 (which is what I expected). This is because the default filter will return every object with a male property for which true is a substring.

To get exact matching behavior (and thereby return 1 object in the aforementioned example) pass true as the 3rd argument to the $filter('filter')(...) call.


Wednesday, August 17, 2016

Chrome Tip - Searching Every/All Source Files At Once

To search through every source file loaded by the currently viewed web page (e.g., the HTML, JavaScript and CSS) open Chrome Dev Tools. From the Sources tab right click topmost node in the tree then choose "Search in all files".

Given that it's accessed via an element in a tree one wonders if it can be used to search through all source files under a given node? A quick test confirms this!

Wednesday, August 10, 2016

Asynchronous Validation In Angular 1.3*

Problem Introduction

One of Angular 1.3*'s more powerful features is its concept of directives. Directives are, in my opinion, what separate Angular from other frameworks and also contributes to its ease of adoption.

If you have worked with HTML at all then you've applied attributes to HTML elements to alter appearance and/or behavior in some way. Why not augment that with the ability to implement custom attributes?

Why Fix This Now?

The need for an application domain specific, server-based auto-complete/type-ahead text input widget crossed an internal, totally subjective threshold of mine. That threshold, N=3, is the point at which I'd like to start consolidating those 3 separate instances into a reusable component because N=4 is rarely far behind N=3 whereas sometimes N=2 is a fluke...

From a framework agnostic perspective this involves providing the user with a textbox for input then, as they type, querying the server and presenting matching options.

How Can Angular Help?

Angular's facility for handling this is its validation pipeline. While it could be done synchronously part of the rationale for an async-heavy framework like Angular is raising the level of interactivity of web applications.

Client side UI frameworks have had support for the behavior for ages (e.g., a ComboBox).

By using the async validation pipeline the user is able to modify other input, resize the browser, etc...  while waiting for results to update. This increases the perceived interactivity of the application.

How Does Angular's Async Validation Work?

Angular's validation framework is exposed through the Form that is published onto $scope by name (e.g., $scope.myForm when the form tag's name attribute is myForm). The details are well described in the Developer Guide/Forms/Custom Validation section.

A Few Considerations To Keep In Mind

  1. Use the custom validation example as your starting point.
  2. The user should be given some indication of progress while the server-side query is executing.
    1. myForm.$pending.myCustomAsyncValidator is defined (e.g., angular.isDefined() returns true) while the operation is executing. Makes an excellent candidate for an ng-show binding.
  3. The promise returned by the custom validator is a little tricky. If the async call succeeds the then-able promise executes any chained callbacks. But it is very likely that you need to inspect the result of the call to determine if the call's input was valid.
    1. To do this, have your custom validator return $q.reject(reason) from the success callback which will trigger rejection.
  4. If validation is failed (e.g., your validator returns $q.reject(reason)) then the error is published to the form property's error object (e.g., myForm.myProperty.$error.myCustomAsyncValidator evaluates to true).
    1. myForm.$valid will also be false.
  5. If validation succeeds (e.g., your validator returns anything other than a rejected promise) then myForm.myProperty.$error.myCustomAsyncValidator evaluates to false.
    1. This doesn't mean that myForm.$valid will be true. All validators (both synchronous and asynchronous) must successfully resolve for myForm.$valid to be true.

Code Snippets


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
angular.module('myModule').directive('myCustomAsyncValidator', ['$q', function($q) {
  //  ... returns a definition object, most omitted for brevity
  return {
    link: function(scope, elm, attrs, ctrl) {
      ctrl.$asyncValidators.myCustomAsyncValidator = function(modelValue, viewValue) {
        
        // ...
        
        // promise executes the server call using viewValue as input, 
        // omitted for brevity
        
        return promise.then(function(data) {
          if (data.failsValidation) { 
            return $q.reject('input not valid');
          } else {
            return angular.copy(data);
          }
        }); // could also handle case where the server call fails to complete here...
      }
    }
  };
}]);

This directive is applied to the HTML input element as follows:

1
<input type="text" name="myProperty" ng-model="data.myModel.myProperty" my-custom-async-validator>