Creating and article list (or Blog) with IBM WCM 8.0

For those of you finding youselves struggling a bit with IBM 's Web Content Manager (8.0) toys, and would like to see a nice and big example, I've posted something on DZone.

For a recent project I was tasked with creating a small Blog platform for IBM Websphere Portal 8.0. The requirements weren't for a full-blown blog, complete with user comments, but rather for a place where authors could create and share business-approved articles.

As this is not a trivial task if you do not know WCM, I decided to document how to do this via a series of five DZone articles. The reason for doing it on DZone is so that it can reach a bit more people. On this blog I'll just list the URLs to those articles if you're interested.

Part I sets the scene and creates the WCM Library;
Part II creates the article authoring and presentation templates, the site area for our articles and discusses Element tags.
Part III creates the list view for the articles.
Part IV adds some authoring tools for external authors.
Part V concludes the series by adding everyting to Portal pages.

The mindset of Agile: why we struggle in enterprises

As a developer I've wondered why we struggle so much to get along with business people in a classical enterprise or big corporate. With the whole "agile" movement, we are now supposed to work a lot closer. So why do we still struggle?

My suspicion is that we think fundamentally differently, like a right-brained artist versus a left-brained bean-counter. In this article I'm exploring these differences to try and discover how this affects agile adoption.

The Developer

We software developers are used to thinking in an agile way. The nature of our business is to think about both the big picture and sweat the details. We need to understand the business needs as well as how to realize it in software.

To be able to do our job, you need to keep on top of technology, and therefore learnt to learn really quickly. In my case, the downside of this is that my memory and attention span became shorter, in no small measure due to the volumes of information my brain is fed daily.

So, put a team of us together and you get a bunch of hyperactive kids who don't like sitting still and has the attention span of a 2-year-old. But we can get the job done, learn really quick, get kicks out of well-written code and new toys, enjoy cool UI design and have a drive to excellence (I hope).

What we struggle with is structure, meetings, lots of Excel spreadsheets or thick requirements docs.

The developer's idea of agile is to adapt often, learn and deliver quickly and to be a bit chaotic. The time-frame for this tend to be one to three weeks max before we lose interest.

The Business person

With a degree in, let's say, financial planning, banking or an MBA, the typical business person is trained in an industry and practices refined in the past 50 years or more. Efficiency, creating order from chaos and planning and the tooling that goes with it is a staple. You learn to follow the rules laid out by your industry. These rules do change, but typically only a small percentage of it, and only once a year.

Put a bunch of business people together and they hold meetings, create plans, hold road-shows, create documentation explaining rules, or listing financial plans.

So essentially generating contracts, and plans to execute these contracts. 

The business person may also use some software to assist them to assist a client or do their work, depending on their role or industry. In general though it is a linear process, well-known and oft-trodden.

What they struggle with is delivering value quickly, with a minimal plan, and postponing details until the latest. They struggle to not write too much and not plan too much.

A business person's idea of agile is to measure and adapt to client needs relatively quickly and to get to market faster than the competition. The time-span is measured in a few months or a financial quarter/year.

The mindset of Agile

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

To paraphrase the agile manifesto, which I believe is about behavior and a mindset and less about a SCRUM or SAFe framework, it is all about collaboration and teamwork, trust, accountability and fail-fast/learn-fast/deliver fast.

So, essentially it fits developers like a glove. Not so much the business person who was trained to do process, documentation, contract negotiation and following plans.

SCRUM and SAFe help mending bridges

SCRUM helps to create a little bit of order in the chaos that is software development and the developer's mind. It helps the business people to plan a little bit at least and understand (by virtue of visual elements like the board, burn-down and demos) where they're at and get a better grip on what they'll get.

SAFe does nothing for developers but lets higher-up business people feel like they're doing agile. Hopefully after a while they will actually be thinking agile and not just do it lip-service.

None of this makes any difference if business do not trust developers, developers don't trust each other, accountability isn't given where it should be and collaboration is still a once-a-quarter meeting.

SCRUM and SAFe then become marketing tools and status symbols instead of assisting an agile way of work.

The enterprise and HR

Often times in an enterprise, HR (and the enterprise as an entity) don't really understand or know what to do with this new way of work.

The company has ceremonies to award individuals and bonuses are given out to (those perceived to be) star players. Quarterly or yearly performance, or "360", reviews revolve around individual performance, and exclude or minimize the key values of teamwork and the agile mindset or expected culture. In essence the company then rewards an individualistic culture, while on paper it states that it is an agile culture.

I believe this is a result of business and HR following the world they know, which is indeed individual-based. HR themselves aren't necessarily schooled in the arts of being agile or the new "startup-style" of work.
Maybe they should also be included in the same SCRUM and SAFE training or agile bootcamps as the rest of us?

Conclusion

By understanding the way we think vs the way business and even HR or the enterprise as an entity thinks, we can be more aware and tolerant of each other and become "agile-ish" enough to satisfy both the chaos of software developent, and the realities that are Business.

In a future post I'm going to list some key points of research on exactly why agile (could) fail, what gets in the way of making it work and a road to get to agile nirvana.

IBM WCM 8.0 - virtual-portal-scoped actions

A challenge we came across when building our REST service for using WCM as a document library was: how do I target a virtual portal? WCM content libraries are created in a virtual portal. This means that, while you're working on the same VP as your content library, you can happily access WCM content via the Java API (see my example code in an earlier article). The problem comes in when you want to access content in a library on one VP, while your client code sits on another VP or are completely run outside of the Portal (as is the case in a REST client).

To make it possible to access content across virtual portals, IBM released a fixpack (CF13) that includes something called a VirtualPortalScopedAction and some additions to the Repository. This API comes standard with Portal 8.5, but for Portal 8.0/WCM 8.0 you need to upgrade to the right fixpack.

Diving in to a sample

I've created a Gist with some sample code, which is listed here.

The code uses the WCM Query API to fetch Category items (from the WCM API) on another virtual portal and library, using a hypothetical portlet. The main players in the code are:

• VirtualPortalScopedAction interface to implement. In the run() method you can access the WCM API as if you are running in the correct virtual portal.
• Repository.generateVPContextFromContextPath which is a CF13+ method to generate/get a context for a virtual portal different than the one you're currently on.
• Repository.executeInVP(VirtualPortalContext, VirtualPortalScopedAction) executes your implementation of VirtualPortalScopedAction on the required virtual portal.

	package example.wcm.vpscopedaction;
	import com.ibm.workplace.wcm.api.Category;
	import com.ibm.workplace.wcm.api.Repository;
	import com.ibm.workplace.wcm.api.VirtualPortalContext;
	import com.ibm.workplace.wcm.api.exceptions.WCMException;
	/**
	* Hypothetical Portlet that runs the query against a VP to fetch Categories.
	* This can be a servlet or REST service as well.
	*/
	public class CategoriesViewerPortlet extends javax.portlet.GenericPortlet {
	@Override
	public void doView(RenderRequest request, RenderResponse response) throws PortletException, IOException {
	try {
	// Get the Repository and generate VP Context from our VP name.
	Repository repository = WCM_API.getRepository();
	VirtualPortalContext vpContext = repository.generateVPContextFromContextPath("virtualPortalName");
	// execute our action on that VP
	FindCategoriesAction findCategoriesAction = new FindCategoriesAction();
	repository.executeInVP(vpContext, findCategoriesAction);
	// extract the info
	List<Category> categories = findCategoriesAction.getCategories();
	request.setAttribute("categories", categories); // or a String[] by accessing Category.getName() or whatever.
	} catch (VirtualPortalNotFoundException e) {
	// log this out somewhere or handle it
	} catch (WCMException e) {
	// log this out somewhere or handle it
	}
	getPortletContext().getRequestDispatcher("some/jsp/path.jsp").include(request, response);
	}
	}

view raw CategoriesViewerPortlet.java hosted with ❤ by GitHub

	package example.wcm.vpscopedaction;
	import java.util.ArrayList;
	import java.util.List;
	import java.util.logging.Logger;
	import com.ibm.workplace.wcm.api.DocumentLibrary;
	import com.ibm.workplace.wcm.api.Repository;
	import com.ibm.workplace.wcm.api.VirtualPortalContext;
	import com.ibm.workplace.wcm.api.VirtualPortalScopedAction;
	import com.ibm.workplace.wcm.api.WCM_API;
	import com.ibm.workplace.wcm.api.Workspace;
	import com.ibm.workplace.wcm.api.exceptions.QueryServiceException;
	import com.ibm.workplace.wcm.api.exceptions.WCMException;
	import com.ibm.workplace.wcm.api.Category;
	import com.ibm.workplace.wcm.api.DocumentLibrary;
	import com.ibm.workplace.wcm.api.Workspace;
	import com.ibm.workplace.wcm.api.exceptions.QueryServiceException;
	import com.ibm.workplace.wcm.api.query.ResultIterator;
	import com.ibm.workplace.wcm.api.query.Selectors;
	import com.ibm.workplace.wcm.api.query.SortDirection;
	import com.ibm.workplace.wcm.api.query.Sorts;
	/**
	* Virtual Portal Scoped Action to demonstrate how to access WCM items on a different Virtual Portal than the one
	* the user is currently in.
	*/
	public class FindCategoriesAction implements VirtualPortalScopedAction {
	private static final String LIBRARY_NAME = "WCM Test library";
	private List<Category> categories = new ArrayList<Category>();
	@Override
	public void run() throws WCMException {
	// Create a Repository and an anonymous workspace (can also use an authenticated workspace, but we'll keep it simple for now)
	Repository repository = WCM_API.getRepository();
	Workspace workspace = repository.getAnonymousWorkspace();
	workspace.login();
	DocumentLibrary library = workspace.getDocumentLibrary(LIBRARY_NAME);
	/* Constuct the query. For the example, we're fetching all Category items in our library, sorted ascending.
	*/
	com.ibm.workplace.wcm.api.query.Query query = workspace.getQueryService().createQuery();
	query.addSelector(Selectors.typeIn(Category.class));
	query.addSelector(Selectors.libraryEquals(library));
	query.addSort(Sorts.byName(SortDirection.ASCENDING));
	query.returnObjects();
	/* Make the categories available to client code.
	* This is needed as the run() method does not return a value.
	*/
	ResultIterator resultIterator = workspace.getQueryService().execute(query);
	categories.clear();
	while (resultIterator.hasNext()) {
	categories.add((Category) resultIterator.next());
	}
	repository.endWorkspace();
	}
	public List<Category> getCategories() {
	return categories;
	}
	}

view raw FindCategoriesAction.java hosted with ❤ by GitHub

REST services best practice

There are different opinions out there on best practices for developing RESTful services. Projects have different needs, developers have different opinions and HTTP has its limitations just like most other technologies.


In this blog I'm listing some of the best practices and guidelines that I could find, sprinkled with my own opinion.

Key API requirements

According to Vinay Sahni's article, there are some key requirements for a good REST API.

1. Use web standards when they make sense. Or the inverse, do not use (certain) web standards for your API if the don't make sense in your context.
2. Your API should be friendly to the developer (your API's client) and be browser-explorable. 
3. It should be simple, intuitive and consistent.
4. It must be efficient and accomplish the task easily.
5. Your REST API must match the domain you want to expose and be flexible to change.

API design best practices

Use plural nouns, not verbs: /books and not /getbooks

REST as an architectural style differs from SOAP webservices in that it works on resources, using http mechanisms like POST to indicate an action to be taken. Instead of the action being described by the API (/getbooks), the action is a combination of the resource (/books) and the http method (GET).

It is important to highlight that we use the plural form. It keeps the API consistent and simple, as you don't need to worry about pluralisation of resources.

JSON vs XML vs...

I use JSON when I'm expecting a web client to use the REST service, typically a javascript client.

XML works well for inter-system contracts or when you really need to be strict on the structure (you can also use JSON schema).

Use other return types based on your needs. Octet stream works for returning documents or images directly.

GET and query parameters should not alter state

You wouldn't have an API method called getBook(bookId) to alter the state of the book, so don't do it for your REST service either. GET and query parameters should not alter the state of the resource.

POST vs PUT

Use a POST when creating a new entry. A call to POST will, every time, create a new resource with a new identifier (if applicable).

PUT on the other hand creates a new entry at first, and updates it thereafter.

POST books/book (with appropriate form data or json)
PUT books/book/123 (note the addition of the ID)

Use sub-resources for relationships: /library/books/

When you need to show sub-relationships, build it in to the URI. The rule of thumb here is to not go deeper than 3 levels with this. If you need to go deeper, maybe create query aliases or provide more direct access to the lower-level resources.

Use HTTP headers to indicate serialization formats

Make use of Content-Type to indicate the (return) type of the content, and Accept to indicate what the client wants to accept.. Some developers modify the Content-Type to include versioning information. Nazar Annagurban shows how you can use Content-Type to indicate versioning (maybe for a minor version): application/myapp.v1+json.

Typical examples:
Accept: application/json;
Content-Type: text/plain

Use HATEOAS (hypermedia as the engine of application state) to assist navigation

Although not necessarily used everywhere, there are a lot of proponents for adding urls to your results to aid the client in navigating your resources. For example, URLs are added as part of your JSON to allow pagination or accessing related resources. Again, use web standards where they make sense.

Provide filtering, sorting, field selection and paging for collections

Make it easy for the client of your API to browse collections of resources.

• filtering: filter resources on some field, for example GET /books?category=horror
• sorting: sort resources in forward or reverse order, for example GET /books?sort=title (sort ascending on title) or GET /books?sort=-title (sort descending on title)
• paging: if the collection is large, allow pagination using offset and limit, for example GET /books?offset=20&limit=20 to get the next 20 books, starting at book number 20. A good idea is to add the URIs (HATEOAS) for paging to navigate previous, next and so on.
• field selection: For resources with lots of fields, allow the client to retrieve a subset of fields, for example GET /books?fields=title,isbn

Version your API in the URL: /myapi/v1/

It is seen as best practice to add a simple version in the URL so it's easy to browse your API in the browser between versions. You can also add the version number in the header (see Nazar Annagurban's article). Some APIs add the major version in the URL and the minor version, or a timestamp, in the header.

Sending multiple parameters (range of values)

When you need to specify a range of values in a query, you have two options:

1. application/x-www-form-urlencoded standard: /books?titles[]=title1&titles[]=title2
2. Widely used JAX-RS-happy standard: /books?titles=title1&titles=title2

So, don't do /books?titles=title1,title2 or something along those lines.

Handle errors with HTTP status codes and return results

When something happens, good or bad, use the HTTP status codes. Also return a payload listing the error. The returned 'error' JSON should be consistent across your API.

Some common uses of Http error codes are listed below.

200 OK - Response to a successful request. Use this whenever your GET, POST, DELETE etc was successful.

201 Created - Response to a POST that resulted in a successful creation. You need to also return a response body pointing to the unique ID or location (url) where the new item can be retrieved.

204 No Content - Response to a successful request that won't be returning a body (like a DELETE request or a GET query with no results). I've used this as well when no results were found in a search query.

304 Not Modified - This means the resource has not been modified since the version specified by the request headers If-Modified-Since or If-None-Match.

307/308 Temporary/Permanent Redirect - Use these if you rolled out a new version of your service and want the client to go there instead (older service is now depricated but still functioning).

400 Bad Request - The request is malformed or something on the client is perceived to be in error.

401 Unauthorized - Authentication is required by the client. After authentication, the request may be executed successfully.

403 Forbidden - This means the client was authenticated but not authorized.

404 Not Found - The requested resource couldn't be found (but may be found in the future).

405 Method Not Allowed - When e.g. a POST is sent to a resource that only supports GET.

410 Gone - See also 307/308: you can use this to indicate your service is now removed, not just deprecated.

415 Unsupported Media Type - If the client does a PUT or POST, with Content-Type that the service does not support. For example, you're posting xml while the service wants JSON.

422 Unprocessable Entity - Used for validation or semantic errors.

429 Too Many Requests - When a request is rejected due to rate limiting.

500 Internal Server Error – When your service has an internal error (some exception you can't handle gracefully), use this code. Your service should trace-log it out but send very little information to the client to avoid exploitation.

For more detail you can have a look at the wikipedia list of http status codes.

Conclusion

Although there are different opinions on the RESTful-ness of a service, the fact is you're OK if

• You think Resources (plural);
• You think Http standards (GET, PUT, Headers, Error codes, queries etc) to be the engine of your service and describe the actions;
• You version your service.

And finally, don't overkill your solution if it's not required.

Using IBM WCM 8.0 Query API - sample code

For our project we needed to make use of IBM Web Content Manager as a 'lightweight' document store, or Library, so users can search for their documents using search criteria. To allow us to do this we create a prof of concept using the Query API.

For the purpose of this blog I'm assuming you have a good understanding of how to create WCM ContentItems, Authoring Templates and the like, and that you're familiar with the terminology. You must also have a licensed Portal 8.0 with WCM 8.0 installation to play with.

Accessing WCM content

IBM Web Content Manager 8.0 has different means of accessing WCM Content:

• Via the WCM REST API (out-of-the-box): WCM comes with a REST API that gives you access to most of the concepts, including content items, site areas and the like. For WCM 8.0 though, no facility is available to access or search by Categories.
• Using the Java API: The Java API allows a lot more flexibility, including searching by categories. It is also a lot faster than the 'chatty' REST service. It includes Workspace 'findxxx' methods, or for more advanced cases, a Query API.
• Using a WCM Content Viewer Portlet with Presentation template: This solution works fine for 'normal' content on the site. We wanted a completely custom UI though, and also surface the items in other standalone applications.

Content (library) requirements for the POC

We created some sample content in WCM to showcase the following:

1. Search for content by keywords, or
2. Search for content by categories, or
3. Search for content by name(s), then
4. Search for or limit results based on custom elements (e.g. text element)

Demo portlet

To make it easy to play with, we created a JSR286 Portlet with a simple JSP, deployed as a WAR directly on a Portal 8.0 environment (same virtual portal/portal as the WCM Library).

The demo code listed at the end of the blog demonstrates the following usage of the WCM Query API and Workspace.

1. Use a WCM Workspace for a specific logged in user;
2. Limit search results to a given WCM Library;
3. Search for Content (Items) by category(ies), keyword(s) or name(s) - these are built-in fields or profile abilities of ContentItems;
4. Filter the results on a custom Element (a text element) value (the Query API doesn't seem to support searching on Element in Documents directly).

Getting the project to work in your environment

Content

Before creating the portlet, be sure to have some WCM ContentItems with keywords, categories and the like available in a WCM Library where the user has access.

WAR

You will need to create a WAR with a JSR286 Portlet, with the appropriate Portal WCM jar (ilwwcm-api.jar), which should come with your Portal installation and be available in RAD or eclipse with Portal 8 Development stubs. So, your project should either have the correct maven dependencies or the Portal 8 libraries attached.

Once set up, you can copy the code into your portlet java class and your own JSP, deploy it and test.

References

WCM Query API:

http://www-10.lotus.com/ldd/portalwiki.nsf/dx/WCM_Query_API

Using the more base Workspace find facilities (code recipes): 

http://saranshibmblog.blogspot.co.za/2012/12/wcm-api-document.html

Code listing

You can get the code as Gist here as well.

	package sample.wcm.query.portlet;
	import java.io.IOException;
	import java.util.ArrayList;
	import java.util.Arrays;
	import java.util.List;
	import javax.portlet.GenericPortlet;
	import javax.portlet.PortletException;
	import javax.portlet.PortletRequestDispatcher;
	import javax.portlet.RenderRequest;
	import javax.portlet.RenderResponse;
	import javax.portlet.ResourceRequest;
	import javax.portlet.ResourceResponse;
	import org.apache.commons.lang.StringUtils;
	import com.ibm.workplace.wcm.api.Category;
	import com.ibm.workplace.wcm.api.Content;
	import com.ibm.workplace.wcm.api.ContentComponent;
	import com.ibm.workplace.wcm.api.DocumentId;
	import com.ibm.workplace.wcm.api.DocumentLibrary;
	import com.ibm.workplace.wcm.api.FileComponent;
	import com.ibm.workplace.wcm.api.ImageComponent;
	import com.ibm.workplace.wcm.api.Repository;
	import com.ibm.workplace.wcm.api.TextComponent;
	import com.ibm.workplace.wcm.api.WCM_API;
	import com.ibm.workplace.wcm.api.Workspace;
	import com.ibm.workplace.wcm.api.exceptions.ComponentNotFoundException;
	import com.ibm.workplace.wcm.api.exceptions.OperationFailedException;
	import com.ibm.workplace.wcm.api.exceptions.QueryServiceException;
	import com.ibm.workplace.wcm.api.exceptions.ServiceNotAvailableException;
	import com.ibm.workplace.wcm.api.query.Disjunction;
	import com.ibm.workplace.wcm.api.query.ProfileSelectors;
	import com.ibm.workplace.wcm.api.query.ResultIterator;
	import com.ibm.workplace.wcm.api.query.Selectors;
	/**
	* A portlet that demos accessing content items with the Query API and doing filtering on Elements.
	* @author RBester
	*
	*/
	public class WcmQueryExamplePortlet extends GenericPortlet {
	private static final String DOCUMENT_LIBRARY = "java-api-test-library";
	private static final String NATIONAL_ID_ELEMENT_NAME = "nationalID";
	@Override
	public void serveResource(ResourceRequest request, ResourceResponse response) throws PortletException, IOException {
	super.serveResource(request, response);
	// comma-delimited list of content item names, category names, keywords and a custom element.
	String contentNames = request.getParameter("contentNames");
	String categoryNames = request.getParameter("categoryNames");
	String keywords = request.getParameter("keywords");
	String nationalIds = request.getParameter("nationalIds");
	//Retrieve the workspace
	Workspace workspace = null;
	Repository repository = WCM_API.getRepository();
	try {
	// Use the logged-in user credentials to log in to the workspace
	workspace = repository.getWorkspace(request.getUserPrincipal());
	workspace.login();
	// use a specific WCM library that is available on the Virtual portal
	DocumentLibrary library = workspace.getDocumentLibrary(DOCUMENT_LIBRARY);
	com.ibm.workplace.wcm.api.query.Query query = workspace.getQueryService().createQuery();
	/*
	* Select all content items for a specific library, with either categories, keywords or names as requested.
	* Use a Disjunction (OR) compound selector to add the relevant Selectors to (categories OR keywords OR names)
	*/
	Disjunction or = addOrClauses(workspace, library, categoryNames, keywords, contentNames);
	// limit to the library and Content, with the specified OR clause.
	query.addSelector(Selectors.libraryEquals(library));
	query.addSelector(Selectors.typeIn(Content.class));
	query.addSelector(or);
	// We're interested in the objects themselves (Document) and not the IDs (DocumentId)
	query.returnObjects();
	/*
	* Filter the results on the provided custom element (TextElement) value.
	* We need to filter, as there isn't a facility in the Query API to search on Document Elements.
	*/
	ResultIterator resultIterator = workspace.getQueryService().execute(query);
	List<Content> content = packageContentInList(resultIterator);
	// apply filter
	content = filterByNationalId(content, nationalIds);
	// print the results as HTML by traversing the Content Elements (previously known as ContentComponent and still that name in the API)
	response.getWriter().write(generateDocumentList(content));
	} catch (ServiceNotAvailableException e) {
	e.printStackTrace();
	} catch (OperationFailedException e) {
	e.printStackTrace();
	} catch (QueryServiceException e) {
	e.printStackTrace();
	} finally {
	if (workspace != null) {
	workspace.logout();
	}
	if (repository != null) {
	repository.endWorkspace();
	}
	}
	}
	private Disjunction addOrClauses(Workspace workspace, DocumentLibrary library, String categoryNames, String keywords, String contentNames) {
	Disjunction or = new Disjunction();
	addCategoriesSelectors(workspace, library, or, categoryNames);
	addKeywordSelectors(or, keywords);
	addNamesSelectors(or, contentNames);
	return or;
	}
	private void addCategoriesSelectors(Workspace workspace, DocumentLibrary library, Disjunction or, String categoryNames) {
	if (StringUtils.isEmpty(categoryNames)) {
	return;
	}
	try {
	List<DocumentId> categories = findCategories(workspace, library, categoryNames);
	or.add(ProfileSelectors.categoriesContains(categories));
	} catch (QueryServiceException e) {
	e.printStackTrace();
	}
	}
	private void addKeywordSelectors(Disjunction or, String keywords) {
	if (StringUtils.isEmpty(keywords)) {
	return;
	}
	or.add(ProfileSelectors.keywordsContain(splitKeywords(keywords)));
	}
	private void addNamesSelectors(Disjunction or, String itemNames) {
	if (StringUtils.isEmpty(itemNames)) {
	return;
	}
	or.add(Selectors.nameIn(splitKeywords(itemNames)));
	}
	/*
	* Loops through the content items and interrogates the ContentComponent (Element). If it's a TextElement (which it should be),
	* the text should match our search criteria.
	*/
	private List<Content> filterByNationalId(List<Content> contentItems, String nationalIds) {
	if (StringUtils.isEmpty(nationalIds)) {
	return contentItems;
	}
	List<Content> toReturn = new ArrayList<Content>();
	for (Content contentItem: contentItems) {
	try {
	ContentComponent contentComponent = contentItem.getComponent(NATIONAL_ID_ELEMENT_NAME);
	if (containsContentText(nationalIds, contentComponent)) {
	toReturn.add(contentItem);
	}
	} catch (ComponentNotFoundException e) {
	System.out.println("component not found: " + e.getMessage());
	}
	}
	return toReturn;
	}
	/*
	* Match TextComponent's text with a comma-delimited list of search terms
	*/
	private boolean containsContentText(String textToScan, ContentComponent contentComponent) {
	if (contentComponent instanceof com.ibm.workplace.wcm.api.TextComponent) {
	String componentText = ((com.ibm.workplace.wcm.api.TextComponent) contentComponent).getText();
	return textToScan.contains(componentText);
	}
	return false;
	}
	/*
	* Casts and adds results to a List for convenience.
	*/
	private List<Content> packageContentInList(ResultIterator resultIterator) {
	List<Content> toReturn = new ArrayList<Content>();
	Object nextItem;
	while(resultIterator.hasNext()) {
	nextItem = resultIterator.next();
	if (nextItem instanceof Content) {
	toReturn.add((Content)resultIterator.next());
	}
	}
	return toReturn;
	}
	/*
	* Generate HTML to be printed
	*/
	private String generateDocumentList(List<Content> content) {
	StringBuilder documentHtml = new StringBuilder();
	for (Content contentItem : content) {
	documentHtml.append(getContentDescription(contentItem));
	}
	return documentHtml.toString();
	}
	/*
	* Tokenize the keywords/search terms
	*/
	private List<String> splitKeywords(String keywords) {
	String [] splitKeywords = keywords.split(",");
	return Arrays.asList(splitKeywords);
	}
	/*
	* To search by categories, we need to first fetch all the categories' IDs, then use those in the query.
	* Typically one could cache the categories so this lookup isn't needed all the time.
	*/
	private List<DocumentId> findCategories(Workspace workspace, DocumentLibrary library, String categories) throws QueryServiceException {
	String[] categoriesSplit = categories.split(",");
	com.ibm.workplace.wcm.api.query.Query query = workspace.getQueryService().createQuery();
	query.addSelector(Selectors.typeIn(new Class[] {Category.class}));
	query.addSelector(Selectors.libraryEquals(library));
	query.addSelector(Selectors.nameIn(categoriesSplit));
	query.returnIds();
	ResultIterator resultIterator = workspace.getQueryService().execute(query);
	List<DocumentId> toReturn = new ArrayList<DocumentId>();
	while (resultIterator.hasNext()) {
	toReturn.add((DocumentId) resultIterator.next());
	}
	return toReturn;
	}
	/*
	* Returns html for ContentItem with text, an ImageComponent/Element and a FileComponent (e.g. pdf).
	*/
	private String getContentDescription(Content content){
	String result = "";
	try {
	String[] componentNames = content.getComponentNames();
	for(int i = 0; componentNames != null && i < componentNames.length; i++){
	ContentComponent contentComponent = content.getComponent(componentNames[i]);
	if(contentComponent instanceof TextComponent){
	result += componentNames[i] + ": " + ((TextComponent)contentComponent).getText() + "<br>";
	} else if(contentComponent instanceof ImageComponent){
	result += componentNames[i] + ": <a href='" + ((ImageComponent)contentComponent).getResourceURL() + "'>Image</a><br>";
	} else if(contentComponent instanceof FileComponent){
	result += componentNames[i] + ": <a href='" + ((FileComponent)contentComponent).getResourceURL() + "'>File</a><br>";
	}
	}
	result += "<br>";
	} catch(ComponentNotFoundException e){
	e.printStackTrace();
	}
	return result;
	}
	@Override
	protected void doView(RenderRequest request, RenderResponse response) throws PortletException, IOException {
	PortletRequestDispatcher dispatcher = getPortletConfig().getPortletContext().getRequestDispatcher("/WEB-INF/jsp/WcmQueryUtilitiesPortlet.jsp");
	dispatcher.forward(request, response);
	}
	}

view raw WcmQueryExamplePortlet.java hosted with ❤ by GitHub

	<%@ page contentType="text/html" pageEncoding="UTF-8" language="java"%>
	<%@ taglib prefix="portlet" uri="http://java.sun.com/portlet_2_0"%>
	<portlet:defineObjects/>
	<form method="post" action='<portlet:resourceURL />'>
	Content name(s): <input type="text" name="contentNames">
	Category(ies): <input type="text" name="categoryNames">
	Keyword(s): <input type="text" name="keywords">
	National IDs(s): <input type="text" name="nationalIds">
	<input type="submit" value="submit">
	</form>

view raw WcmQueryExamplePortlet.jsp hosted with ❤ by GitHub

Making a success of microservices

I've been reading quite a lot on microservices architecture, especially the team and organisational culture required for this approach to be a success. More specifically, I found it just as interesting to see when NOT to use this approach, than when it's appropriate. 

So without further ado, let's run through some findings and thoughts, followed up by reading material for your perusal.

Essential rules of thumb

In order for you, your team and your organisation to even think about, let alone succeed, with microservices, think about the following.

1. Know when it's appropriate to apply it: microservices architecture isn't a one-size-fits-all solution and can actually hurt you, if you don't apply it with eyes wide open. Sometimes monoliths are quite appropriate;
2. Know your team culture: your team need to be willing to do what it takes, and be patient. Microservices architecture ride heavily on team maturity, tooling and techniques to be able to deliver it effectively;
3. Know your organisational culture: Conway's Law states "organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations". Essentially, your organisation's structure and culture should be able to, or willing to, give autonomy to teams and have less governance. The more red tape, the less likely microservices are to succeed; 
4. Be willing to trial-and-error, and grow into it rather than big-bang: companies like Google, Amazon and more didn't start off with microservices as an architectural approach; they all started with a monolithic approach and grew into it, to solve real architectural and business challenges. 

Tips on your road to success

To get you on your way, here are some tips and tricks gathered from the experts (see the list at the bottom). 

Make sure it's a fit for your project(s)

Before you apply this architectural style to a project, make sure:

• you're not applying it because it's the latest cool kid on the block. Sometimes a monolithic approach is a perfectly good solution.
• it will meet your client's needs in terms of performance, functionality and effort required;
• you have a real customer problem or business case that is hard to solve in a monolith, and apply microservices architecture there.

Team and culture

Your team must be willing to own a product (one or more microservices) top-to-bottom and be mature enough to be able to do the following.

• Decide on the technology to use with little or no governance, and be autonomous as far as possible;
• Have cross-cutting skills in the team, so the team can address from the database right through to the UI if need be;
• Your team should either be willing to, be moving towards, or already be doing Continuous Deployment. Microservices imply you're going to have lots of them, so you really need to be efficient with how you develop, test and deploy them;
• The team must be held (and want to be held) accountable for that product and own it right up to production support (DevOps).

Deploy, Refactor, Reuse Recycle

Microservices need to be boxed (see Domain Driven Design's bounded context) and simple to reason about.

1. You must be willing to retire, add and modify them separately and often. This means you need to be on top of your versioning game;
2. Reuse other teams' microservices before you build your own. This means strong collaboration between teams;
3. Use Domain Driven Design, paying special attention to the Bounded Context principle, where appropriate, to make sure your microservices don't end up being semi-monoliths.

So, when does it make sense to use microservices as an architectural approach?

I found some tips from Adrian Trenman (Gilt) very informative. Use microservices as an architectural approach when

1. you can isolate a piece of domain functionality that a single service can “own”;
2. the service can fully own read and write access to its own data store;
3. multiple teams are contributing to a monolithic system but keep on stepping on each other’s toes;
4. you want to implement continuous deployment;
5. you favor an emergent architecture rather than atop-down design.

Conclusion

The take-home from this article, if  you remember nothing else, is that you need to be mature as a team before embarking on the microservices journey. It requires trust from your team and organisation for it to be successful.

In addition, you also need to understand that it is a tool in the toolbox of architectures, and not a magic bullet. 

Lastly, don't rush it. Be OK with trial-and-error and make sure your organisation understands that.

References

Thank you to the following resources and contributors to those resources. I highly recommend you read up on these articles if you want to know more.

 InfoQ microservices architecture e-book where most of my info comes from. People like Eric Evans, Martin Fowler and Randy Shoup (plus some others from Gilt) were contributors.

From monolyth to microservices by Randy Shoup (slideshare).

InfoWorld's article on how to succeed with microservices.

SmartBear's article on what is microservices architecture.

How to JUnit test post-login code in Websphere Application Server 7+

Quite recently I wrote a WAS 7 utility that needed to access the JEE security Subject (read more on JEE Security) and its private credentials. The project, an auditing tool, accesses the Subject via IBM's WSSubject feature and uses some of its methods.

While trying to unit test it I ran into several problems, especially around trying to mock container-provided classes like Subject or LoginModules, and especially getting WSSubject to work.

Solving post-login unit test nightmares

When dealing with container-provided services like JAAS and the security model it can be very hard to write unit tests that exercise the code out-of-container, aka automated-build-tool-friendly tests. You can most probably start up an embedded container, but even so, setting up security to the point where it will populate your Subject can be tricky.

After a lot of research and trial-and-error, I created a test utility that allows you to:

1. Let WSSubject return correct values for getRunAsSubject(), getCallerSubject() and getCallerPrincipal();
2. Allow a unit test to add test private credentials to the Subject before testing a code unit that uses it;
3. Allow a unit test to add a custom LoginModule and/or test custom LoginModules without the need of a container (not even a lightweight one);
4. Allow the whole lot to run nicely from e.g. Jenkins in an automated build process.

Get to the code already

The samples contain 3x java classes:

• A LoggedinTestContext with helpers to facilitate the login in out-of-container unit testing scenarios;
• A MockLoginModule to show how to test JAAS Modules and/or add stuff to a Subject;
• Sample unit test (LoggedinSubjectTest) to illustrate the LoggedinTestContext's usage and what it can provide.

The example code is not a full project as it depends on your own setup and is specific to your IBM WAS installation. The test data is also samples added as private credentials, but you can add anything your unit of code requires out of the Subject.

Dependencies needed

The code samples require you to (either in maven dependencies or elsewhere) have the following libraries available.

 WAS (7+) dependencies

All the jars below are relative to your WAS install root (e.g. IBM/Websphere/AppServer).

• /lib/bootstrap.jar
• /plugins/com.ibm.ws.runtime.jar
• /runtimes/com.ibm.ws.admin.client_<version>.jar
• /lib/j2ee.jar
• /plugins/com.ibm.ws.emf.jar
• /plugins/org.eclipse.emf.ecore.jar
• /plugins/org.eclipse.emf.common.jar
• /java/jre/lib/ibmcfw.jar

Where  <version> is your WAS version, e.g. 7.

JUnit dependencies

For the rest of the dependencies, the test code sample uses normal junit dependencies (junit mockito etc).

Code

I've added comments in the code so you should be able to follow along. I included the Gist listing in the blog, but you can it also get it from here.

Happy hunting!

	package thecodingglass.testing.postloginsamples;
	import static org.unitils.reflectionassert.ReflectionAssert.assertLenientEquals;
	import java.util.HashMap;
	import java.util.Map;
	import org.junit.Test;
	/**
	* Shows how you can grab/verify that your stuff is populated on the Subject. In the real world the unit tests will test some unit of code
	* that internally interrogates the Subject.
	*/
	public class PostLoginSampleTests {
	@Test
	public void verifyPrivateCredentialsAreBlank() {
	// log in using a default user
	new LoggedinTestContext().login();
	// WSSubject should return correct value
	assertNotNull(WSSubject.getCallerSubject()));
	assertNotNull(WSSubject.getRunAsSubject()));
	assertEquals(WSSubject.getCallerPrincipal(), LoggedinTestContext.DEFAULT_USERNAME);
	// you could test some sample code that needs a Subject and specific private credentials here.
	}
	@Test
	public void privateCredentialsMustBePopulatedOnSubject {
	LoggedinTestContext loggedinContext = new LoggedinTestContext();
	Map<String, String> testCredentials = loggedinContext.createDefaultPrivateCredentials();
	loggedinContext.withCustomCredentials(testCredentials, LoggedinTestContext.DEFAULT_USERNAME).login();
	// you can also use loggedinContext.withDefaultCredentials().login(); if you're OK with the defaults.
	Set<Object> privateCredentialSet = WSSubject.getCallerSubject().getPrivateCredentials();
	Map<String, String> myPrivateCredentials = (Map<String, String>) privateCredentialSet.get(0);
	assertLenientEquals(testCredentials, myPrivateCredentials);
	}
	}

view raw LoggedinSubjectTest.java hosted with ❤ by GitHub

	package thecodingglass.testing.utils;
	import static org.mockito.Mockito.mock;
	import static org.mockito.Mockito.when;
	import java.security.Principal;
	import java.util.HashMap;
	import java.util.HashSet;
	import java.util.Map;
	import java.util.Set;
	import javax.security.auth.Subject;
	import javax.security.auth.callback.CallbackHandler;
	import javax.security.auth.login.AppConfigurationEntry;
	import javax.security.auth.login.Configuration;
	import javax.security.auth.login.CredentialExpiredException;
	import javax.security.auth.login.LoginContext;
	import javax.security.auth.login.LoginException;
	import momentum.retail.audit.messages.LoggedinAuditMessage.PrivateCredentialKey;
	import com.ibm.websphere.security.WSSecurityException;
	import com.ibm.websphere.security.auth.CredentialDestroyedException;
	import com.ibm.websphere.security.auth.WSSubject;
	import com.ibm.websphere.security.cred.WSCredential;
	import com.ibm.ws.security.core.ContextManagerFactory;
	/**
	* Creates a LoggedinTestContext where you can simulate user login.
	*
	* Some sample usage:
	* <ul>
	*
	* <li><code>new LoggedinTestContext().login(); // logs you in with no username or private credentials</code>
	* <li><code>new LoggedinTestContext().withDefaultCredentials().login(); //logs you in with 'derek' username and example private credentials as the SalesLoginModule will provide.</code>
	* <li>
	* <code>
	* LoggedinTestContext loggedinContext = new LoggedinTestContext();
	* Map<String, String> defaultCredentials = loggedinContext.createDefaultPrivateCredentials(); // modify or add to defaultCredentials your own values
	* loggedinContext.withCustomCredentials(defaultCredentials, "derek").login(); // log in with your modified credentials
	*
	* </code>
	* <li> <code>new LoggedinTestContext().withCustomCredentials(<your own Map here>, <own username>).login();// log in with your own stuff.</code>
	* </ul>
	*
	* You can then expect the following to be valid:
	* <ul>
	* <li>WSSubject.getCallerPrincipal() returns your username;
	* <li>WSSubject.getCallerSubject() gets you a populated Subject;
	* <li>WSWSubject.getRunAsSubject() gets you the same subject.
	* </ul>
	* @author RBester
	*
	*/
	public class LoggedinTestContext {
	public static final String DEFAULT_USERNAME = "joedirt";
	public static final String DEFAULT_IP_ADDRESS = "10.1.1.1";
	public static final String DEFAULT_UNIQUE_ID = "SF_GUID_123";
	public static final String DEFAULT_IMPERSONATING_USER = "no-one";
	public static final String DEFAULT_IMPERSONATING_USER_GUID = "no-one123";
	public static final String DEFAULT_SESSION_ID = "joedirt_session_id";
	private static final String LOGIN_MODULE_NAME = "thecodingglass.testing.utils.MockLoginModule";// The Login process creates an instance of this via reflection.
	private LoginContext context;
	private Set<Object> privateCredentials = new HashSet<Object>();
	private Set<Object> publicCredentials = new HashSet<Object>();
	private Subject subject = null;
	public LoggedinTestContext() {
	}
	/**
	* Will add default private credentials to the Subject (see static Strings in LoggedinTestContext)
	* @return Returns this instance for chaining.
	*/
	public LoggedinTestContext withDefaultCredentials() {
	return withCustomCredentials(createDefaultPrivateCredentials(), DEFAULT_USERNAME);
	}
	/**
	* Will add your custom private credentials (as Map) and username to the Subject.
	* @param privateCredentialMap ustom private credentials. This can be accessed from the Subject via the subject.getPrivateCredentials (returns Set) grabbing the first object you can find.
	* @param username Your custom username.
	* @return Returns this instance for chaining.
	*/
	public LoggedinTestContext withCustomCredentials(Map<String, String> privateCredentialMap, String username) {
	WSCredential userCredential = mock(WSCredential.class);
	try {
	when(userCredential.getSecurityName()).thenReturn(username);
	} catch (CredentialExpiredException e) {
	e.printStackTrace();
	} catch (CredentialDestroyedException e) {
	e.printStackTrace();
	}
	privateCredentials.add(privateCredentialMap);
	publicCredentials.add(userCredential);
	return this;
	}
	/**
	* Expose creation of the default private credentials if you want to use this as basis for your test expectations or custom input.
	* @return Returns a Map with default credentials.
	*/
	public Map<String, String> createDefaultPrivateCredentials() {
	Map<String, String> credentialMap = new HashMap<String, String>();
	credentialMap.put(PrivateCredentialKey.IMPERSONATING_USERNAME.name(), DEFAULT_IMPERSONATING_USER);
	credentialMap.put(PrivateCredentialKey.IMPERSONATING_USERGUID.name(), DEFAULT_IMPERSONATING_USER_GUID);
	credentialMap.put(PrivateCredentialKey.IP_ADDRESS.name(), DEFAULT_IP_ADDRESS);
	credentialMap.put(PrivateCredentialKey.UNIQUE_ID.name(), DEFAULT_UNIQUE_ID);
	credentialMap.put(PrivateCredentialKey.SESSION_ID.name(), DEFAULT_UNIQUE_ID);
	return credentialMap;
	}
	/**
	* Trigger the login process.
	*/
	public void login() {
	try {
	init();
	context.login();
	} catch (LoginException e) {
	e.printStackTrace();
	}
	}
	private void init() throws LoginException {
	CallbackHandler handler = mock(CallbackHandler.class);
	subject = createSubject();
	try {
	// the business-end: create LoginContext and configure it with modules.
	context = new LoginContext("AuditLoggerTest", subject, handler, new Configuration() {
	@Override
	public AppConfigurationEntry[] getAppConfigurationEntry(String arg0) {
	AppConfigurationEntry[] toReturn = new AppConfigurationEntry[] {
	// Our login module. You can add more here.
	new AppConfigurationEntry(LOGIN_MODULE_NAME, AppConfigurationEntry.LoginModuleControlFlag.REQUIRED, new HashMap<String, Object>())
	};
	return toReturn;
	}
	});
	} catch (LoginException e) {
	e.printStackTrace();
	}
	}
	private Subject createSubject() {
	Subject subject = new Subject(true, new HashSet<Principal>(), publicCredentials, privateCredentials);
	try {
	ContextManagerFactory.getInstance().setCallerSubject(subject);
	WSSubject.setRunAsSubject(subject);
	} catch (WSSecurityException e) {
	e.printStackTrace();
	}
	return subject;
	}
	}

view raw LoggedinTestContext.java hosted with ❤ by GitHub

	package thecodingglass.testing.utils;
	import java.util.Map;
	import javax.security.auth.Subject;
	import javax.security.auth.callback.CallbackHandler;
	import javax.security.auth.login.LoginException;
	import javax.security.auth.spi.LoginModule;
	/**
	* A JAAS Module that you can use to test post-login situations. The LoginModule is allowed to modify credentials etc.
	* Refer to the LoggedinTestContext: this class is instantiated by the javax.security.auth.login.LoginContext via reflection.
	*/
	public class MockLoginModule implements LoginModule {
	/**
	* Simply prints out the provided private credentials, but a login module can also add credentials. These will then be available
	* on the Subject.
	* @param subject The JAAS subject in the process of logging in.
	* @param callbackHandler JAAS chain callback handler (see the JEE spec for more on this).
	* @param sharedState State shared between login modules.
	* @param options Options get passed in to the LoginModule via the container on initialization.
	*/
	@Override
	public void initialize(Subject subject, CallbackHandler callbackHandler, Map<String, ?> sharedState, Map<String, ?> options) {
	if (subject.getPrivateCredentials() != null && !subject.getPrivateCredentials().isEmpty()) {
	Map<String, ? extends Object> privateCredentials = (Map<String, ? extends Object>) subject.getPrivateCredentials().toArray()[0];
	System.out.println("MockLoginModule: Subject private credentials: " + privateCredentials.toString());
	} else {
	System.out.println("MockLoginModule: No private credentials.");
	}
	}
	/*
	* Simply allow login to proceed. You can return false if login shoud fail based on what happened on initialize.
	*/
	@Override
	public boolean login() throws LoginException {
	return true;
	}
	/**
	* All modules executed with true; commit.
	*/
	@Override
	public boolean commit() throws LoginException {
	return true;
	}
	/*
	* Rollback as a commit or something else failed.
	*/
	@Override
	public boolean abort() throws LoginException {
	return true;
	}
	/*
	* On the logout leg. True if logout was OK, false otherwise.
	*/
	@Override
	public boolean logout() throws LoginException {
	return true;
	}
	}

view raw MockLoginModule.java hosted with ❤ by GitHub