Many exciting things have been happening internally on the build tools for Sencha frameworks.
Some weeks ago we decided to reorganize our development of Sencha Cmd around a very exciting proof-of-concept: a framework-aware, JavaScript compiler (the new name for “SDK Tools” a.k.a. “Sencha Command”).
The gains from just the first few optimizations were so promising that we decided to accelerate the release of this technology and focus the Sencha Cmd team on bringing the compiler out of the lab and into your applications as quickly as possible. Along the way, of course, we will be fixing as many of the bugs reported against version 2 as we can.
We have also added numerous new guides on how to use Sencha Cmd with your Ext JS and Sencha Touch applications, and you can find them here.
As usual, please provide any feedback you may have regarding Sencha Cmd and the documentation through our forum.
Learn the how to build and test web applications using Sencha Cmd at SenchaCon 2013. Register today!
Table of Contents
The View From Inside Sencha
For several reasons we have consolidated many of the tools in Sencha Cmd into Java. Since Java has always been a system requirement for doing builds, this may go unobserved by most. This transition has really helped us leverage perhaps the most exciting highlight to share: the recent (and large) upgrade to our internal Continuous Integration (CI) system. All of this newly added capacity not only allows our product teams to continue to expand their test coverage, but also allows us to add Sencha Cmd to the mix.
With Java we are able to easily monitor detailed code coverage generated by the CI system for all of the functionality in Sencha Cmd. We can also watch the performance trends for some of the more expensive operations to keep your build cycle times down. I am happy to report that, at the present time, we have approximately 70% code coverage (by line) and expect to significantly increase that number as we audit the uncovered areas of the code throughout the beta process.
OK, so enough talk about us! On to the good stuff. :)
Introducing Sencha Cmd V3
The following is a high-level summary of the major features of Sencha Cmd V3. I will expand on several of these. This should be somewhat familiar to those who have used the previous releases, but many things are all new or significantly expanded.
- Code generation tools to generate entire applications and extend those applications with new MVC components, for both Ext JS and Sencha Touch.
- A framework-aware, JavaScript compiler that knows the semantics of Sencha frameworks
and can produce minimal footprint builds from your source. In the future, the compiler
will optimize many of the high-level semantics provided by Sencha frameworks to reduce
load-time of your applications. - Sencha Mobile Packager provides a way to seamlessly “package” your web app in a native shell and instantly making it app-store-ready.
- Powerful code selection tools for tuning what is included in your application’s final
build, determine common code across pages and partition shared code into “packages”—all
using high-level set operations to get builds exactly as you want them. - Workspace management to assist in sharing code between pages or applications.
- Image extraction to convert CSS3 features (such as border-radius and linear-gradient)
into sprites for legacy browsers. - Flexible configuration system allows defaults to be specified for command options at
the application or workspace level, or across all workspaces on a machine. - Robust logging to help understand the inner workings of commands and facilitate
troubleshooting. - Integration with Apache Ant.
- Code generation hooks that can be specific to one page or shared by all pages in the
workspace (for example, to check coding conventions or guidelines as new models are
generated). - Reliance on significantly fewer third-party, native executables. This can be very helpful in large organizations that need IT approval for each one. Gone are HammerJS, Jsdb and Node.
Compiler vs. Builder
In previous versions of tooling, builds were basically just file concatenation. For Ext JS, this was driven by a “JSB” file that contained the list of source files to include in their proper order. This file was maintained in part by hand and in part using tools that would run your application to determine its requirements. Things are similar in Sencha Touch where app.json
is used.
Using the new compiler, all of this information is read directly from your application’s source code and the source code of the framework. One of the main benefits of this global view of the source is that the compiler can provide helpful, code-level diagnostic messages.
To get the complete view of your Ext JS applications, the compiler can now also process script tags in a section of your markup that you specify. In future releases, this will enable Sencha Cmd to inject conditional logic directly in your markup to eliminate a round-trip before loading other dependencies.
The high-level, application commands have all been adapted to use the compiler, so you won’t need to change anything if you are using commands like sencha app build
. If your application was not produced with sencha generate app
or moving your code into that structure is not an option for you, then there are new commands that expose the compiler’s features directly.
Ext JS Code Generation
A feature that Sencha Touch users have had for some time is code generation. Sencha Cmd now provides the same level of support for application, model, view and controller code generation for Ext JS. This can help accelerate new projects by removing the need to hand-craft all the boilerplate. It also provides consistency for team members.
The process starts like so:
sencha -sdk /path/to/extjs generate app MyApp /path/to/app
Various other commands, like sencha generate model
and sencha generate controller
, will extend the generated application. When you are ready, sencha app build
produces the build.
One of the most useful features provided to generated Ext JS applications is that Sencha Cmd maintains a “bootstrap” file containing important metadata about your application’s source code. This file is included in the generated HTML file and takes care of setting up paths for the dynamic loader as well as informing the loader of all classes in your application. Formerly, this metadata was only available for framework code and it is what enables the following code to work:
Ext.define(‘MyApp.foo.Bar’, {
requires: [
‘Ext.grid.*’, // this worked previously
‘MyApp.util.*’, // but now this works!
],
// …
});
Further, you won’t need to call Ext.Loader.setPath
anymore since all that information is maintained in the bootstrap file.
The only issue to contend with is that the metadata contained in the bootstrap file becomes stale when you add, remove or rename classes or namespaces. All that is required to update your application’s metadata is to run this command from your application folder:
sencha app refresh
This functionality is only supported for Ext JS today, and will likely be extended to Sencha Touch in subsequent releases. To see how you can use the compiler to generate a bootstrap file yourself. see the Single-Page Ext JS Apps guide.
Using Workspaces To Share Code
A common request from folks building large applications containing multiple pages is to have a convenient way to share code between pages. Even more common is to be able to share code between your Ext JS and Sencha Touch front-ends.
The new workspace feature enables organizations to manage shared code and configuration across multiple pages. While a workspace is little more than a folder that contains a .sencha
folder (with content understood by Sencha Cmd), a workspace enables Sencha Cmd to find common configuration information for all pages contained anywhere inside the workspace.
The most common thing to do is to set a default “classpath” for the compiler so that all compile commands will find shared code. Once a folder is added to the workspace classpath, it will automatically be searched when compiling any page in the workspace.
Sencha Mobile Packaging
Sencha Cmd includes the latest and greatest version of the Sencha Mobile Packager. This is the new name for the “Native Packager” to align it with the exciting and new Sencha Desktop Packager included in the Sencha Complete: Team bundle.
The Sencha Mobile Packager is installed as part of Sencha Cmd, so once you have Sencha Cmd V3 installed, you are ready to go. A welcome relief for some: this installation no longer requires administrative permission.
The Compiler
To build an Ext JS application, say in the structure produced by Sencha Architect, you need run only a command like this:
sencha compile -classpath=app.js,app,ext/src page -in=app.html -out=build/index.html
There are only a couple requirements to meet for the compiler to be able to do its work.
First, you must use requires
where appropriate to avoid any kind of synchronous loading during development. This is really no different than the current tools and recommendations, so this may not require any changes to your application.
Second, you will need to add some new directives to your markup (HTML or ASPX or JSP or PHP or …) file:
These comments allow the compiler to identify the scripts at the “root” of your application (in this case, app.js
) as well as ignore the “bootstrap” part of the framework you use in development. This file includes a minimal amount of the framework and primes the dynamic loader. In a compiled application, all of these same files are pulled in by the dependency process.
This section of the markup file will be replaced by a single script tag to the generated all-classes.js
file that will be placed in the same folder as the markup file.
A Deeper Look
Some folks occasionally need to perform custom builds of the framework. For example, they may have applied a patch to the source. In Ext JS 4.1.1a and 4.1.2a, we now ship a build script that enables this command:
sencha ant build
This will build the standard set of “ext*.js” files from the contents of the src
folder. To see the compiler commands required to build the framework from source, you can examine the provided build.xml
file.
Custom builds can be made using the compiler directly. To illustrate, let’s create a build that we have occasionally contemplated adding: ext-all-no-charts.js
. Now with the compiler, this can be accomplished with this command:
sencha compile -classpath=ext/src exclude -namespace Ext.chart and concat -out ext-all-no-charts-dev.js and -option debug:false concat -out ext-all-no-charts-debug.js and concat -yui -out ext-all-no-charts.js
Breaking this down, we can see several advanced features of Sencha Cmd at work. In particular the use of “command chaining”. The and
command conjunction tells Sencha Cmd to continue processing arguments against the current command category. In the above example, that is the compile
command.
The first part loads up the compiler and tells it the folder for source (i.e., the “classpath”). In this case, we are pointing at an extracted Ext JS SDK in a folder named ext
:
sencha compile -classpath=ext/src
The above will automatically include all files in the classpath as part of the “current set” of files. Since we don’t want charts, we exclude the namespace:
exclude -namespace Ext.chart and
We then concatenate out (in their proper order) to ext-all-no-charts-dev.js
:
concat -out ext-all-no-charts-dev.js and
We chain the above command using and
, so we are now continuing at the compile
command. We reconfigure the options to remove debug code.
-option debug:false
The set of files is the same, but their content will be somewhat different and we concatenate the result to ext-all-no-charts-debug.js
.
concat -out ext-all-no-charts-debug.js and
Another and
to continue back at the compile
command, so we again concat, but this time we enable the YUI Compressor.
concat -yui
Writing the compressed output to ext-all-no-charts.js
.
-out ext-all-no-charts.js
Theme Building for Ext JS
Building image sprites for custom themes has always been somewhat complex and error prone. Because this is critical for supporting legacy browsers while taking advantage of CSS3 features such as border-radius and linear-gradient, we have revamped this process.
In Sencha Cmd, this now begins with an HTML page that you can open in a CSS3-compatible browser like Firefox or Chrome. So you can then make sure your theme is rendering properly before generating image sprites.
The architecture for theme pages is designed to make including custom components a simple task. Over time, we hope to see custom component providers supply “theme building” definition files to go along with their components. This would enable their users to easily incorporate those components into custom themes.
Getting the Beta
Hopefully all of that has you excited to try out the beta of Sencha Cmd. We really depend on, and greatly appreciate, your feedback to help us ensure that Sencha Cmd meets your needs.
For Sencha Touch 2.1 Beta 3, you just need to download the latest Sencha Touch SDK and Sencha Cmd. For Ext JS applications, you will need either Ext JS 4.1.1a or Ext JS 4.1.2a. These releases has very few changes beyond what is required to work with Sencha Cmd. This chiefly amounts to some added compiler directives (just comments) to key files, plus a sencha.cfg
file required for Sencha Cmd.
You can download the Ext JS 4.1.1a from the Ext JS product download page.
Ext JS 4.1.2a is available from the Sencha Support Portal.
The latest beta refreshes can be found on the forum here.
Conclusion
We are very excited to hear about your experience with the new Sencha Cmd. Tell us the good and the bad (so we can fix it!). We want this version to enhance and streamline your development experience with Sencha frameworks!
Do the words about Theme Building mean that soon we will be able to no only create ext-custom.js but also ext-custom.css that excludes the CSS for ExtJS components that I don’t use? Getting rid of some CSS burden should help improve the performance of ExtJS apps with highly customized styles! Looking forward to see that…
We’ve been super excited about this for a while!! This tool is going to help us with our build automation. Good job Don & Team!
Cannot wait to try this out! Looks awesome :)
Very cool. Looking forward to trying it out.
Great ! Thanks for sharing !
I guess the 6+ months of almost no doc for Sencha SDK/Tools was worth it now. :-P If this somehow lets me integrate the app build into our CI, I’ll be very happy indeed. :-)
Can you give some more details into why did you move away from NodeJS? I was expecting the opposite in Sencha CMD evolution…
I translated it into Japanese.
http://www.xenophy.com/javascript/3653
Provision: Japan Sencha User Group
http://www.meetup.com/Japan-Sencha-User-Group/about/
Great news! Good job, Sencha. I’m looking forward to using this in the future (once I’m convinced that it’s stable… I’ve gotten cut by some sharp edges on previous betas of the Sencha Command…). Also, I’m glad that you’re reducing the number of third-party dependencies. Personally, I was ok with the node.js dependency, but it’s encouraging that Sencha listened to the feedback from enterprise users for whom it was a problem. Settling on Java makes sense. It should help with enterprise acceptance and integration, and for indie devs and open-source geeks like me, there’s enough excitement around recent developments in the JVM world — Scala, Clojure, Play Framework, etc. — that I think it shouldn’t be too hard to accept. Thanks!
@Kazuhiro – thanks :)
@Dmitry –
Good question. Obviously we love JavaScript, so it is not anything negative there. The primary reason for the shift was the tool chain. If running at the command prompt on all platforms is the goal (vs, say, in a browser) and you want world class development tools, Java has a lot going for it. It produces blazingly fast code while supporting refactoring. providing detailed code coverage (just check the “I want it” box) and other ways as well. This has really allowed us to focus on solving the real problems for users of Cmd.
I hope that answers your question.
@Dmitry –
We are looking at better integration of theme building, but this release just updated how image sprites are produced and provided a structure for automation of that as well as the ability to extend what is being themed.
@Don,
thanks for replying! So currently there is no SASS/Compass integration in Sencha.CMD?
Speaking of the tool chain, what about some essential tools like the mentioned SASS/Compass (that provides the nice `watch` feature), jshint, qunit/jasmine? I want a single build command to trigger all these in different configuration.
Also, a little side question: is there any documentation on “conditional compilation” for js that removes stuff like this (as often seen in ExtJS sources)?
//
Ext.Error.raise(‘bummer!’);
//
Are there any other tags that can be preprocessed?
@Dmitry –
Confirmation of one of those reasons from a forum user:
“It is so much faster than SDK Tools 2.0.0 beta3. It even makes you think something is wrong :)”
:)
The PATH is not well setted on OS X Lion (10.7.4)
Actually the .profile is updated with the path to Sencha Cmd,
but this should be in .bash_profile !!
@Jay. Jay, I am getting your MEAP updates for your ExtJS 2nd Edition book. Will you be adding a chapter about this?
Thanks,
Murray
What about sencha complete members ? Where can I download Ext JS 4.1.2a ?
I’m currently playing with Cmd, but I got some issues:
First: It seems to be that the default Maximum Heap Space for Java (configured in sencha.cfg) is a little bit to low for Ext JS applications. However, I’ve used a bigger app. Because I’m aware of Java and Heap settings, that wasn’t a problem for me to find the parameter and the correct settings. But for the others, I’d suggest a hint what to do when a “Heap error” will occor.
Second: Replaying your example, I’ve noticed that I actually cannot use “YUI”. Using it, the Builder applies all tasks before but then rejects it with “No such property : yui” (on OS X 10.8.2). Do I need any additional requirement?
Third: While playing with the commands, I’ve noticed that following arguments are equal
and concat -out file.js
and concat -out=file.js
Additionally, I’m a little bit confused about the chaining.. or let’s say: I’m not aware what it _can_ do and what it _cannot_ do. I would suggest a) something like a Backus–Naur Form to make clear what is chainable. And b) some specials like concat which seems to “drop” a file while piping the content to the next sub command in the chain.
Last but not least: What about a descriptive configuration file, let’s say like the Maven’s pom.xml? That have not to be an XML… JSON is good enough.
PS: What does “option:false/true” mean? And, furthermore, what can I do as a developer to make fun of that option?
@Peter
You can download Ext JS 4.1.2a from the support portal: http://support.sencha.com
Well, I see you have already done things describing chaining and dedicated configuration file.. hidden in your docs http://docs.sencha.com/ext-js/4-1/#!/guide/command_advanced
@Mitchel
Well, I can not. Only available dowload I can see is Sencha complete 1.0.0.19 with a date Sep 11th, 2012. Does this package contain ExtJS 4.1.2a ? I can see only 4.1.2.
@Peter
I see 4.1.2a:
Ext JS 4.1.2a Sep 18th, 2012
This using a regular user, not my admin account.
@Mitchell
I guess, because I have Sencha Complete subscription I can see only complete package. I can see touch and extjs nightly builds but no Extjs nor touch libraries are avaialable for download separately like 4.1.2a. Everything is in the currently outdated package. Another thing.. Should I download whole package with every minor update ? I don’t get this.
@Jan –
I’d love to help you out with those questions but the forum (https://staging.sencha.com/forum/forumdisplay.php?8-Sencha-Cmd) would be a better place and it would be most helpful to break up your questions/comments into separate posts to help maintain a clear conversation.
That said – 1) we will likely increase the heap size; 2) check out http://docs.sencha.com/ext-js/4-1/#!/guide/command_advanced for details on command chaining and shortest-unique-prefix matching; 3) for issues with running particular commands it would really help to see the exact command and output, but again, in the forum is best; 4) yuicompressor is in the box – we use it to build the framework – so there must be something amiss on the command.
Thanks for all the feedback though – great to see people are trying it out :)
@Jan – it seems you found the guide already … just responding top-down ;)
@Murray, Absolutely!
Ask Don. I’ve been pestering him for weeks. It’s the last great dependency for Sencha Touch 2.0 and Ext JS 4.1.1 in action ;)
@Peter –
Please let us know what kinds of packaging are helpful to you – we are always listening to how our customers want to consume the pieces of the Sencha Complete offerings. It sounds like you would like access to the whole bundle *and* the individual products for download?
@Murray – confirmed … weeks :O
Will this be integrated into Sencha Architect as well?
When integrated with Sencha Architect?
When is it part of Sencha Architect (SA)? The new structure generated is not compatible with SA ( not like v2 beta.). An example is the location of app.js is now inside app and not at the same level.
@All –
We hear you on that :) the Sencha Architect team is working closely with the Sencha Cmd team on integration and leverage. The file location difference is just a small difference in Sencha Cmd’s classpath so any applications generated by SA can be handled by Sencha Cmd.
@Don
Separated packages would be great. Should I contact someone in sencha about this ?
@Peter – you just did :)
Great work guys! :)
Can the compiler be used to filter out configs, properties, events and methods that are not required/being utilized within a code base?
Regards
MrSparks
@MrSparks –
We are looking at deeper optimizations, but that level of scrubbing is pretty advanced. At the moment, we are focusing on class system optimizations to make those common operations faster.
I’m having trouble figuring out how to build an app that includes a vendor library.
I get “Failed to resolve dependency Sch.panel.SchedulerTree for file ExtCalendar.view.Tree”
And, I have yet to find an example or guide or forum post that helps me resolve.
My post is here:
https://staging.sencha.com/forum/showthread.php?243748-Integrating-Bryntum-Ext-Scheduler-With-Cmd-generated-Ext-app
at last the sencha mobile packager needed a revamp and some good docs because it is a nightmare to use
@Peter – I believe the individual packages are now available in the support portal. :)
First , My appreciations for consant updates , That’s good new for the Sencha texhnologies.
I have a concern , When we use the command tool to generate the Application code , Can we open the same Generated Code in the Sencha Architect ?
Ridiculous and convoluted… command mentioned here don’t match items for doc which doesn’t work either…