Dec 12, 2014

ListView is probably the most common android component, but it has to be implemented correctly to provide a  better user experience.

In this post, I will give a few suggestions on how you can achieve the near optimum list performance.  They are mentioned below:

  1. Access Data Incrementally:- Load only the data you need. I know this is a general principle to reduce memory footprint of your application, but even so, this can and must be done.  The ListView class can listen for scroll updates, you can implement the OnScrollListener interface to do so.  When the user scrolls to the bottom of the list, then load the incremental data. This can be done independent of the data source, whether it be a REST web service or the SQLite database. In this post, I am going to show you how you can handle this when your view is backed by the data from the database.  Take a look at this abstract class Endless Scroll Listener which implements the OnScrollListener. To integrate this with your data source and ListView all you  have to do is extend from this class as shown below.
      // The listener implementation
    listener= new EndlessScrollListener() {
    @Override
    public void onLoadMore(int page, int totalItemsCount) {
    // Triggered only when new data needs to be appended to the list
    // Add whatever code is needed to append new items to your AdapterView
    customLoadMoreDataFromApi(page);
    // or customLoadMoreDataFromApi(totalItemsCount);
    }
    private void customLoadMoreDataFromApi(int page) {
    //Progress dialog to show with loading text
    pd.setTitle(R.string.loadingText);
    pd.show();
    Bundle arg = new Bundle();
    //here pageSize is a constant
    //How many records to fetch
    limit = pageSize;
    //From which offset
    offset = (page - 1) * pageSize;
    //add these to the parameters that will be passed to the loader
    arg.putInt("limit", limit);
    arg.putInt("offset", offset);
    //add custom arguments

    //Instantiate or restart the loader
    Loader loader = getActivity().getSupportLoaderManager().getLoader(LOADER_ID);
    if (loader != null && !loader.isReset()) {
    getActivity().getSupportLoaderManager().restartLoader(LOADER_ID, arg, _self);
    } else {
    getActivity().getSupportLoaderManager().initLoader(LOADER_ID, arg, _self);
    }
    }
    };
    //set on scroll listener for the listview
    lstView.setOnScrollListener(listener);

    SQLite provides limit and offset clause, so when the user scrolls downwards, we restart the loader with the new offset parameter. Below I have mentioned the methods involved in loader callback implementations.

    @Override
    public Loader<Object> onCreateLoader(int id, Bundle args) {
    Loader loader= new SimpleAbstractListLoader(getActivity().getApplicationContext(),id,args);
    return loader;
    }
    @Override
    public void onLoadFinished(Loader<Object> arg0,
    Object arg1) {
    //here article list is an array list that contains the data
    //if it is initial just create a new list else append the data to the existing list
    if(articleList==null||adapter==null){
    //FeedArticleSummary is a POJO class
    articleList=new ArrayList<FeedArticleSummary>();
    articleList.addAll((ArrayList<FeedArticleSummary>) arg1);
    //Custom Adapter for the listview that extends the ArrayAdapter
    adapter = new ArticleListAdapter(this.getActivity(),
    R.layout.article_list_layout,articleList);
    lstView.setAdapter(adapter);
    }
    else{
    articleList.addAll((ArrayList<FeedArticleSummary>)arg1);
    adapter.notifyDataSetChanged();


    }
    //load finished so dismiss the progress dialog.
    pd.dismiss();
    }

    Now the loader class SimpleAbstractListLoader just needs to pass the limit and offset parameters to a database utility class which will apply limit and offset to the SQL Query to fetch the data. i.e rawQuery+whereClause+OrderBy + limitandOffsetClause.


    Here one thing should be noted that order by clause is not optional and is required as the data needs to be fetched in a predefined order so that the offset clause can fetch the correct data.




  2. Use a ViewHolder and know that views can be recycled:  A ViewHolder pattern avoids repeated calls to findViewById() which is used to lookup the views in layout.  It is just a static class  whose instance holds the component views inside the tag field of the Layout.  Secondly, List Views can be recycled, so its a best practice to reuse the views and populate them with new data,  instead of inflating them  again.  These two when combined together will make the scrolling of the list very smooth and fast.  Here the latter concept is even more important than the former, as inflating a view is a costly operation and should definitely be avoided.  So, if you are using ArrayAdapter as a data source for your list, this is how your getView() method should look like.

       @Override
    public View getView(int position, View convertView, ViewGroup parent) {
    ViewHolder holder;
    // Get the data item for this position
    //custom pojo which hold the reference to the data
    FeedArticleSummary articleSummary = getItem(position);
    if(articleSummary==null ){
    return null;
    }
    // Check if an existing view is being reused, otherwise inflate the view
    if (convertView == null) {
    convertView = mInflater.inflate(R.layout.article_list_layout, null);
    holder=new ViewHolder();
    holder.text=(TextView) convertView.findViewById(R.id.textViewArticleListFragment);
    holder.imageButton=(ImageButton)convertView.findViewById(R.id.favoriteImageArticleList);
    holder.checkBox= (CheckBox) convertView.findViewById(R.id.selected);
    //Here the holder is set as a tag field in the layout
    convertView.setTag(holder);
    }
    else{
    holder=(ViewHolder)convertView.getTag();
    }
    //set the data here
    //for example
    // holder.checkBox.setChecked(true);
    ....
    }


    And here is the ViewHolder class.

       static class ViewHolder {
    TextView text;
    CheckBox checkBox;
    ImageButton imageButton;
    }



  3. Do not set the layout_height and layout_width property to wrap_content:  Definitely avoid this because getView() will be invoked a number of times,  to determine the height and width for the views that are to be drawn. Instead use fill_parent or use fixed height and width, so that there are no unnecessary invocations of the getView() method.


  4. If you know which row has been modified call getView() instead of notifyDataSetChanged().  Often, a user might modify a single item in  a list and you might be required to make some UI modification for that item. For example, if a user selects a list item to view the details, you might change the font style of that particular listitem, to show that the item has been read. In this case it is much faster to just invoke the getView() for that item  rather than calling notifyDataSetChanged().

    @Override
    public void onListFragmentItemClick(ListView l, View view, int position,
    long id) {

    ListView lstView = l;
    //adapter reference
    ArticleListAdapter articleAdapter = (ArticleListAdapter) l
    .getAdapter();
    //accessor for the underlying list
    ArrayList<FeedArticleSummary> articleSummaries=articleAdapter.getArticleSummaries();
    FeedArticleSummary articleSummary = articleAdapter.getItem(position);
    if(articleSummary.getArticleRead()==null||articleSummary.getArticleRead().equals("N")) {
    //update the underlying data source for the list item
    articleSummary.setArticleRead("Y");
    int visiblePosition = lstView.getFirstVisiblePosition();
    View selectedChildView = lstView.getChildAt(position - visiblePosition);
    //update the data in the DB using a loader
    SQLiteCursorLoader loader = new SQLiteCursorLoader(this, fdb.getDbHelper(), null,
    null);
    ContentValues cValues = new ContentValues();
    cValues.put(FeedSQLLiteHelper.COLUMN_ARTICLE_READ, "Y");
    loader.update(FeedSQLLiteHelper.TABLE_ARTICLES, cValues, "_id = ? ", new String[]{articleSummary.getArticleId()});
    //call getview
    lstView.getAdapter().getView(position, selectedChildView, lstView);
    ...
    }

    and in your getView method handle the font style change

    ...  
    if(articleSummary.getArticleRead()==null|| articleSummary.getArticleRead().equals("N")){
    holder.text.setTypeface(holder.text.getTypeface(),Typeface.BOLD);
    }
    else{
    holder.text.setTypeface(holder.text.getTypeface(),Typeface.ITALIC);
    }
    ...




Note: Android has introduced RecyclerView, which forces you to use the ViewHolder as a best practice, this was not enforced earlier in ListView.  



I might have missed on few other optimizations, so kindly do provide your feedback.

Posted on Friday, December 12, 2014 by Unknown

Oct 30, 2014

If you want to integrate Lucene with your android application, this post will get you started.  Lucene provides you with a wide range of searching options  like  Fuzzy Search, wildcard search, etc.  So, you can use this in your android application, if you want to provide search option over your custom data model.

In the code shown below searches will be near real time as I am passing IndexWriter instance to it, so IndexReader will be created using the passed IndexWriter instance.  Also, as creation of IndexWriter and SearcherManager is expensive, so the best place to initialize them is in the application class.

Initialization: The application class which initializes the IndexWriter and SearcherManager.

public class FeedReaderApplication extends Application {

public static final SearcherManager getSearcherManager(){return searcherManager;}
public static final IndexWriter getIndexWriter(){return indexWriter;}
private static SearcherManager searcherManager=null;
private static IndexWriter indexWriter=null;

@Override
public void onCreate() {
//pick the properties from user preferences
SharedPreferences preferences = PreferenceManager.getDefaultSharedPreferences(getApplicationContext());
Analyzer analyzer= new SimpleAnalyzer(Version.LUCENE_41);

IndexWriterConfig config=new IndexWriterConfig(Version.LUCENE_41,analyzer);
//pick the buffer size from property
String memorySize=preferences.getString("lucene_memory_size","5.0");
config.setRAMBufferSizeMB(Double.valueOf(memorySize));
config.setOpenMode(IndexWriterConfig.OpenMode.CREATE_OR_APPEND);
//create index on external directory under lucene folder
File path= new File(getApplicationContext().getExternalFilesDir(null),"lucene");
try {
Directory directory= FSDirectory.open(path);
indexWriter=new IndexWriter(directory,config);
boolean applyAllDeletes = true;
//no need to warm the search
searcherManager = new SearcherManager(indexWriter, applyAllDeletes, null);
} catch (IOException e) {
Log.e(tag,"Error occurred while opening indexWriter/SearcherManager"+ e.getMessage(),e);
}
}

}


Now, In the example application I am using Sqlite database to store the Feed data, but titles are being analyzed and stored in the lucene index. Also, I am using the SimpleAnalyzer rather than standard analyzer as the StandardAnalyzer does stop word filtering before storing the terms which is not going to work out for us as the user might search with stop words and find no matches.



public class LuceneSearchUtil {
private static final String tag = LuceneSearchUtil.class.getName();

public LuceneSearchUtil() {
}
//insert articles id,title and feedid
public static void insertArticleDocument(ContentValues contentValues) {
try {
IndexWriter writer = FeedReaderApplication.getIndexWriter();
Document document = new Document();
//don't analyze id field, store as such
Field idField = new StringField(FeedSQLLiteHelper.COLUMN_ID, String.valueOf(contentValues.get(FeedSQLLiteHelper.COLUMN_ID)), Field.Store.YES);
document.add(idField);
//analyze the url field so textfield
Field titleField = new TextField(FeedSQLLiteHelper.COLUMN_ARTICLE_TITLE, String.valueOf(contentValues.get(FeedSQLLiteHelper.COLUMN_ARTICLE_TITLE)), Field.Store.YES);
document.add(titleField);
Field feedId= new StringField(FeedSQLLiteHelper.COLUMN_ARTICLE_FEED_ID,String.valueOf(contentValues.get(FeedSQLLiteHelper.COLUMN_ARTICLE_FEED_ID)), Field.Store.YES);
document.add(feedId);
writer.addDocument(document);
} catch (IOException e) {
Log.e(tag, "Unable to add document as " + e.getMessage(), e);
}
}

//searching the articles searchterm is passed and broken down into individual terms
public static ArrayList<String> searchAndGetMatchingIds(String searchTerm) {
ArrayList result=new ArrayList<String>();
//get the searchermanager
SearcherManager searcherManager = FeedReaderApplication.getSearcherManager();
IndexSearcher indexSearcher = null;

indexSearcher = searcherManager.acquire();
//split on space
String[] terms= searchTerm.split("[\\s]+");
//multiple terms are to be searched
SpanQuery[] spanQueryArticleTitle=new SpanQuery[terms.length];
int i=0;
for (String term:terms){
//wildcardquery
WildcardQuery wildcardQuery=new WildcardQuery(new Term(FeedSQLLiteHelper.COLUMN_ARTICLE_TITLE,term.toLowerCase()));
spanQueryArticleTitle[i]=new SpanMultiTermQueryWrapper<WildcardQuery>(wildcardQuery);
i=i+1;
}
//no words between the typed text you could increase this but then performance will be lowered
SpanNearQuery spanNearQuery1=new SpanNearQuery(spanQueryArticleTitle,0,true);
TopDocs topDocs=null;
try {
//execute topN query
topDocs = indexSearcher.search(spanNearQuery1, ProjectConstants.LUCENE_TOP_N);
if(topDocs!=null){
for(ScoreDoc scoreDoc:topDocs.scoreDocs){
Document document= indexSearcher.doc(scoreDoc.doc);
String id= document.get(FeedSQLLiteHelper.COLUMN_ID);
result.add(id);
}
}
} catch (IOException e) {
e.printStackTrace();
}
finally {
try {
searcherManager.release(indexSearcher);
} catch (IOException e) {
Log.e(tag,"Exception while releasing Index Searcher "+e.getMessage(),e);
}
}

return result;
}

//sample delete method

public static void deleteArticlesByFeedId(String feedId){
IndexWriter indexWriter = FeedReaderApplication.getIndexWriter();
TermQuery query=new TermQuery(new Term(FeedSQLLiteHelper.COLUMN_ARTICLE_FEED_ID,feedId));
try {
indexWriter.deleteDocuments(query);
} catch (IOException e) {
Log.e(tag, "Unable to delete document as " + e.getMessage(), e);
}
try {
indexWriter.commit();
} catch (IOException e) {
Log.e(tag, "Unable to commit changes " + e.getMessage(), e);
}
}
}


Note the search method in the above code. It is going to split the query that user passed into individual terms and then search each of those terms by using SpanNearQuery with a word distance of 0, which means that whatever user has typed must be matched without a word gap. For example if user types:  “Sweet Orange ” then those two terms will be matched only if there is no word between them in the article title.  Also note that Lucene returns top matching results, so when you pass these id’s to your database for retrieving the actual data from the database, you must make sure that the returned data is in that order.  Here is the relevant snippet from the AsyncTaskLoader.



 @Override
public List<FeedArticleSummary> loadInBackground() {
//query the searchterm
ArrayList<String> ids= LuceneSearchUtil.searchAndGetMatchingIds(searchTerm);
ArrayList results=new ArrayList();
//returns all the articles that match
HashMap<String,FeedArticleSummary> map= fdb.getEntriesForFeedByIds(ids);
//order them
if(map!=null){
for(String id:ids){
if(map.get(id)!=null){
results.add(map.get(id));
}

}
}

return results;
}


Now, all you need to do is invoke the loader to query and load the data when the user uses the SearchView in your application.  Here are the implemented methods for the SearchView.OnQueryTextListener.



 @Override
public boolean onQueryTextSubmit(String queryText) {
//let's set a threshold
if (queryText!=null&&queryText.trim().length() > 5) {
Loader loader = getActivity().getSupportLoaderManager().getLoader(ProjectConstants.LOADER_ARTICLES_SEARCH);
if (loader != null && !loader.isReset()) {
getActivity().getSupportLoaderManager().restartLoader(ProjectConstants.LOADER_ARTICLES_SEARCH, args, _self);
} else {
getActivity().getSupportLoaderManager().initLoader(ProjectConstants.LOADER_ARTICLES_SEARCH, args, _self);
}
if(!pd.isShowing()){
//show the progressdialog
pd.show();
}
}
return true;
}
@Override
public boolean onQueryTextChange(String s) {
//not handling this
return false;
}




Now in onloadfinished method just replace the data in your arrayadapter and you are set. That's it now you have integrated lucene into your application. 



device-search

Posted on Thursday, October 30, 2014 by Unknown

Oct 5, 2014

android_search_view 

In the last post, I had explained how you can use Retrofit API to consume Feedly’s feed search API. In this post, I will cover the integration of a SearchView with the feed search API. 

To integrate SearchView with the custom data source, you need to implement the following  interfaces  in the fragment or activity.

  • SearchView.OnQueryTextListener: This handles firing a query with the data provider as the user starts typing in the SearchView.
  • SearchView.OnSuggestionListener: This handles click or selection of the suggested result from search.

The code for the implemented methods of these interfaces is show below:-

//for onQueryTextListener
@Override
public boolean onQueryTextSubmit(String s) {
if (s.length() > 2) {
loadData(s);
}
return true;
}

@Override
public boolean onQueryTextChange(String s) {
if (s.length() > 2) {
loadData(s);
}
return true;
}

//for OnSuggestionListener

@Override
public boolean onSuggestionSelect(int position) {
Cursor cursor = (Cursor) searchView.getSuggestionsAdapter().getItem(position);
String feedName = cursor.getString(4);
searchView.setQuery(feedName, false);
searchView.clearFocus();
return true;
}

@Override
public boolean onSuggestionClick(int position) {
Cursor cursor = (Cursor) searchView.getSuggestionsAdapter().getItem(position);
String feedName = cursor.getString(4);
searchView.setQuery(feedName, false);
searchView.clearFocus();
return true;
}


 



The layout for the  customized list item which will appear when the user starts typing in the SearchView is shown below:-



<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
android:orientation="horizontal" android:layout_width="match_parent"
android:layout_height="match_parent">
<ImageView
android:id="@+id/icon_feed"
android:layout_width="32dp"
android:layout_height="32dp"
android:layout_centerVertical="true"
android:layout_centerHorizontal="true"
android:layout_alignParentStart="true"
android:contentDescription="@string/ImgContentDescription"
/>
<TextView
android:id="@+id/feed_url_text"
android:layout_width="wrap_content"
android:layout_height="23dp"
android:paddingLeft="5dp"
android:layout_toRightOf="@id/icon_feed"
android:paddingTop="0dp"
android:paddingBottom="2dp"
/>
<LinearLayout
android:id="@+id/subscriber_layout"
android:orientation="horizontal"
android:layout_below="@+id/feed_url_text"
android:layout_toRightOf="@id/icon_feed"
android:layout_width="wrap_content"
android:layout_height="25dp">
<ImageView
android:src="@drawable/icon_user_light"
android:paddingLeft="5dp"
android:layout_width="24dp"
android:layout_height="24dp" />
<TextView
android:id="@+id/subscriber_count"
android:text="@string/feed_name"
android:layout_width="wrap_content"
android:paddingTop="2dp"
android:paddingLeft="2dp"
android:layout_height="23dp" />
</LinearLayout>

</RelativeLayout>


As, I am using a custom layout for the list item that appears in the search, So I have extended the SimpleCursorAdapter to  bind the view to the underlying data as shown below.



public class SearchFeedResultsAdaptor extends SimpleCursorAdapter {
private static final String tag=SearchFeedResultsAdaptor.class.getName();
private Context context=null;
public SearchFeedResultsAdaptor(Context context, int layout, Cursor c, String[] from, int[] to, int flags) {
super(context, layout, c, from, to, flags);
this.context=context;
}

@Override
public void bindView(View view, Context context, Cursor cursor) {
ImageView imageView=(ImageView)view.findViewById(R.id.icon_feed);
TextView textView=(TextView)view.findViewById(R.id.feed_url_text);
TextView subscribersView=(TextView)view.findViewById(R.id.subscriber_count);
ImageTagFactory imageTagFactory = ImageTagFactory.newInstance(context, R.drawable.rss_icon);
imageTagFactory.setErrorImageId(R.drawable.rss_icon);
ImageTag tag = imageTagFactory.build(cursor.getString(2),context);
imageView.setTag(tag);
FeedReaderApplication.getImageManager().getLoader().load(imageView);
textView.setText(cursor.getString(4) + " : " + cursor.getString(1));
subscribersView.setText(cursor.getString(3));


}
}


In this adapter class, I am binding the layout components to the elements returned by cursor in the overridden bindView method . I am also using the novoda image loader to load the icon image from the URL returned by Feedly service.



Since, I am using a fragment , so the code to initialize the SearchView  is placed in the  OnActivityCreated method as shown below.



 @Override
//list of columns
public static String[] columns = new String[]{"_id", "FEED_URL", "FEED_ICON", "FEED_SUBSCRIBERS", "FEED_TITLE"};

public void onActivityCreated(@Nullable Bundle savedInstanceState) {
super.onActivityCreated(savedInstanceState);
searchView = (SearchView) getView().findViewById(R.id.searchFeedView);
searchView.setOnQueryTextListener(this);
searchView.setOnSuggestionListener(this);
mSearchViewAdapter = new SearchFeedResultsAdaptor(this.getActivity(), R.layout.search_feed_list_item, null, columns,null, -1000);
searchView.setSuggestionsAdapter(mSearchViewAdapter);
}


The loadData method which is invoked from the onQueryTextChanged  method loads the Feedly search results as shown below:-



 private void loadData(String searchText) {
//specify endpoint and build the restadapter instance
RestAdapter restAdapter = new RestAdapter.Builder()
.setEndpoint("http://feedly.com")
.build();
//Now use restadapter to create an instance of your interface
FeedlySuggestionsService searchService = restAdapter.create(FeedlySuggestionsService.class);
//populate the request parameters
HashMap queryMap = new HashMap();
queryMap.put("query", searchText);
//implement the Callback method for retrieving the response
searchService.searchFeeds(queryMap, new Callback<FeedlyResponse>() {
@Override
public void success(FeedlyResponse feedlyResponse, Response response) {
MatrixCursor matrixCursor = convertToCursor(feedlyResponse.getResults());
mSearchViewAdapter.changeCursor(matrixCursor);
}

@Override
public void failure(RetrofitError error) {
Log.e(tag, error.toString());
}
});
}


If you notice the highlighted lines in the above method, you will see that the result returned by the REST service is in List format. Now, as the SearchView accepts only CursorAdapter as its SuggestionAdapter, so we have to convert the List into a Cursor object.  To do so, I have created a  convertToCursor method which iterates the List to return a MatrixCursor object.  After the conversion is done, I am just replacing the existing cursor of the Suggestion Adapter with this new value.  The method to convert the list to MatrixCursor is shown below:-



 private MatrixCursor convertToCursor(List<FeedlyResult> feedlyResults) {
MatrixCursor cursor = new MatrixCursor(columns);
int i = 0;
for (FeedlyResult feedlyResult : feedlyResults) {
String[] temp = new String[5];
i = i + 1;
temp[0] = Integer.toString(i);

String feedUrl = feedlyResult.getFeedId();
if (feedUrl != null) {
int index = feedUrl.indexOf("feed/");
if (index != -1) {
feedUrl = feedUrl.substring(5);
}
}
temp[1] = feedUrl;
temp[2] = feedlyResult.getIconUrl();
temp[3] = feedlyResult.getSubscribers();
temp[4] = feedlyResult.getTitle();
cursor.addRow(temp);
}
return cursor;
}


The above method takes the column names for the Cursor from a static String array as shown above the onActivityCreated  method.



That’s it ! now you have a SearchView  integrated with a REST service.

Posted on Sunday, October 05, 2014 by Unknown

Recently, I noticed a poor implementation of dynamic query in our application environment.  As the query was complex and its where clauses were generated dynamically, the developer chose not to implement the bind variables.  Now, some of you will be aware of the disadvantages of not using bind variables, but i have mentioned them below just to recap:-

  • It is a security risk as it allows SQL injection attack
  • It causes a major bottleneck on database as every time a search is submitted, the query will need to be parsed and for this the database acquires a latch.  A latch is like a synchronization code, so at a time only a single query might acquire the latch. thus, if you are parsing the query regularly, you are slowing down your database unnecessarily. 
  • Every time the query is parsed for different values a new statement will be created and stored in shared SQL area, so you have added memory problems to the list of issues.

 

Now that we have seen the issues, the solution then is to use the bind variables.

The problem:  Search was complex and developer did not know how many different clauses will be applied at runtime as the where clause depended upon the field values entered in the criteria.  We could use named bind variables but what happens if the search is executed again ?

The solution:  In case we use named bind variables and add them to the where clause, executing the query again with different criteria will cause JBO exception, because the bind variable will be there but it obviously would not have any value. To fix this, the simple solution then is to remove all bind variables at the start of your query and then add them depending upon the conditions. The code snippet below shows how to handle this.

//In the backing bean

public void doSearch(ActionEvent ae){
DCBindingContainer bc=(DCBindingContainer)BindingContext.getCurrent().getCurrentBindingsEntry();
DCIteratorBinding iteratorBinding=bc.findIteratorBinding("YOUR_VO");
ViewObject voImpl=iteratorBinding.getViewObject();
//Get all the bind variables that we had added
AttributeList whereClauseParams=voImpl.getNamedWhereClauseParams();
//Get their names
String []attribNames=whereClauseParams.getAttributeNames();
for(String attribName: attribNames){
voImpl.removeNamedWhereClauseParam(attribName);
}
StringBuilder whereClause=new StringBuilder();
//dummy
whereClause.append("where 1=1 ");
//now append named bind variables to query
if(empId==1){
//a like comparison bind variable
whereClause.append("and empName like upper(:bEmpName)||'%'");
}
//define the created named bindvariable
vo.defineNamedWhereClauseParam("bEmpName",empName,null);
//invoke am's method to execute the query ?
.....
// In AM
query=query+whereClause;
//execute our defined query
voImpl.setFullSqlMode(ViewObjectImpl.FULLSQL_MODE_AUGMENTATION);
voImpl.setQuery(query);
voImpl.executeQuery();


   



That’s it now you can run your dynamic queries without bugging your DBA.

Posted on Sunday, October 05, 2014 by Unknown

Oct 4, 2014

Retrofit is a type-safe rest client for java/android that one can use for easily consuming rest services. It supports both synchronous and asynchronous retrieval. In this post, I will explain an example which shows how you can integrate it with your android application for consuming rest services asynchronously.  The example will show its integration for doing feed search with Feedly. Feedly’s basic feed search is free, you can find the document here Feedly Search Service. 

Setting up Retrofit : -

I am now using android studio instead of eclipse as android studio is based on IntelliJ Idea platform and performs much better in every aspect.  In your build add the following Gradle dependency, rest will be managed by Gradle.

 compile 'com.squareup.retrofit:retrofit:1.6.1'


About Feedly’s feed search API :-



Feedly's  search feed API has the following request signature. GET /v3/search/feeds It accepts the following three parameters: query Required string search query. Can be a feed url, a site title, a site url or a #topic count Optional number number of results. default value is 20 locale Optional locale hint the search engine to return feeds in that locale (e.g. “pt”, “fr_FR”).



A sample request response is shown below:-



Request:



GET feedly.com/v3/search/feeds?query=ramannanda.blogspot.com



Response:



{"queryType":"term","results":[{"deliciousTags":["adf"],"feedId":"feed/http://ramannanda.blogspot.com/feeds/posts/default","language":"en","subscribers":23,"title":"Technical Essentials","velocity":0.0,"lastUpdated":1402601340000,"website":"http://ramannanda.blogspot.com/","score":2.299999952316284,"estimatedEngagement":5,"description":"Java, ADF, Identity Management, Fusion Middleware, Linux,  Counter Strike 1.6, BSD, Windows, \nProgramming, Search Engines","scheme":"TEXT:BASELINE:ORGANIC_SEARCH","contentType":"longform","partial":false,"twitterScreenName":"ramannanda9","twitterFollowers":75,"facebookUsername":"technicalessentials9911","facebookLikes":3,"iconUrl":"http://storage.googleapis.com/site-assets/WkQvKWtcgur8yMCOJbBWti9mkZPUEZGutfEwCkYH6Pc_icon-1484cef868f","visualUrl":"http://storage.googleapis.com/site-assets/WkQvKWtcgur8yMCOJbBWti9mkZPUEZGutfEwCkYH6Pc_visual","coverUrl":"http://storage.googleapis.com/site-assets/WkQvKWtcgur8yMCOJbBWti9mkZPUEZGutfEwCkYH6Pc_cover","coverColor":"C0DEED"}],"related":["adf"],"scheme":"subs.0"}


Using Retrofit:



Now we need to setup the service interface which can be used to call this service and implement a callback to which the results will be assigned.  



Search Interface:



public interface FeedlySuggestionsService {
@GET("/v3/search/feeds")
void searchFeeds(@QueryMap Map<String, String> options, Callback<FeedlyResponse> cb);
}


The interface is pretty easy to understand.




  • It targets search feeds service as explained above.


  • The searchFeeds method accepts a map of key-value request parameters.


  • It also specifies a parameter that is of callback interface, which we need to implement in an activity or fragment to  use the results.


  • Notice that FeedlyResponse is a custom class and retrofit will parse the JSON and populate the results.  Pretty neat! 



The POJO classes for response are mentioned below:



The FeedlyResponse class:-



public class FeedlyResponse {
public FeedlyResponse(){
}

public List<FeedlyResult> getResults() {
return results;
}

public void setResults(List<FeedlyResult> results) {
this.results = results;
}

private List<FeedlyResult> results;

}


The FeedlyResult class.



public class FeedlyResult {

private String feedId;
private String title;
private String coverUrl;
private String visualUrl;
private String iconUrl;

public String getSubscribers() {
return subscribers;
}

public void setSubscribers(String subscribers) {
this.subscribers = subscribers;
}

private String subscribers;
public String getFeedId() {
return feedId;
}

public void setFeedId(String feedId) {
this.feedId = feedId;
}

public String getTitle() {
return title;
}

public void setTitle(String title) {
this.title = title;
}

public String getCoverUrl() {
return coverUrl;
}

public void setCoverUrl(String coverUrl) {
this.coverUrl = coverUrl;
}

public String getVisualUrl() {
return visualUrl;
}

public void setVisualUrl(String visualUrl) {
this.visualUrl = visualUrl;
}

public String getIconUrl() {
return iconUrl;
}

public void setIconUrl(String iconUrl) {
this.iconUrl = iconUrl;
}


}


If you see the JSON response, you will see that  FeedlyResponse class maps to the response object that is returned and List<FeedlyResult> object maps to the results array. As you can see this is pretty cool, because retrofit  will  populate only the parameters that you specify.



In your activity/fragment:-



You just need to implement a method which fetches the data as shown below.



private void loadData(String searchText) {
//specify endpoint and build the restadapter instance
RestAdapter restAdapter = new RestAdapter.Builder()
.setEndpoint("http://feedly.com")
.build();
//Now use restadapter to create an instance of your interface
FeedlySuggestionsService searchService=restAdapter.create(FeedlySuggestionsService.class);
//populate the request parameters
HashMap queryMap=new HashMap();
queryMap.put("query",searchText);
//implement the Callback<T> interface for retrieving the response
searchService.searchFeeds(queryMap, new Callback<FeedlyResponse>() {
@Override
public void success(FeedlyResponse feedlyResponse, Response response) {
//convert list to cursor. more on this later
MatrixCursor matrixCursor= convertToCursor(feedlyResponse.getResults());
mSearchViewAdapter.changeCursor(matrixCursor);
}

@Override
public void failure(RetrofitError error) {
Log.e(tag, error.toString());
}
});
}


I think the comments on the method should suffice, but let me explain what’s happening in the above method.




  • First, we are using the builder pattern to build the RestAdapter. Here, we are specifying the endpoint to use. 


  • We use the restadapter to create an instance for the interface we had created.


  • Implement the callback interface’s success and failure methods.


Notice that we did not have to parse the JSON to map the parameters to our POJO class, it was done automatically and asynchronous capability was added just by declaration of the callback parameter.



Have fun and let me have some REST!

Posted on Saturday, October 04, 2014 by Unknown

Oct 3, 2014

Android has  a different paradigm for application development. So , In this post, I will  explain a few of those concepts (as I understand) that lie at its core.

  1. Cursor:  To draw parallels,  a cursor object is just like a ResultSet in JDBC. It is used for iterating over the underlying data and just like a JDBC ResultSet object, it must also be closed.  The following snippet shows how to iterate over the  result set.
    try {
    cursor = getDatabase().query(tableName,
    columns, whereClause, whereArgs, groupBy, having, orderBy);
    if (cursor != null) {

    while (cursor.moveToNext()) {
    cursor.getString(cursor.getColumnIndexOf("COLUMN_NAME_TO_SELECT"));

    }
    }
    } catch (Exception e) {
    Log.e("[" + tag+ "] ", "Exception Occured while fetching data", e);
    throw e;

    } finally {
    // make sure to close the cursor
    cursor.close();

    }



  2. Loader :  if you are aware of swing development, then you will know that the best practice is to perform long running tasks such as running a query, fetching data from a service, etc. on a separate thread, this is even more important to follow in android because if you  don’t your app will just crash.  A loader is basically what you use to perform those long running tasks .  So this is what you basically have to do,  In your activity or fragment implement the Loader callback methods,  which would be invoked when the loader has finished performing its long running tasks.  The long running tasks are performed in the loader class’s  loadinbackground method.  


    /**
    The loader class
    */
    public class ArticleLoader extends AsyncTaskLoader<Map<String,String>> {

    /**
    * This is where the articles are loaded
    */
    @Override
    public Map<String,String> loadInBackground() {
    //Perform the data fetch here
    }

    }





    //the fragment class where you create the loader and wait for the loader to do its job
    public class ArticleFragment extends Fragment implements
    LoaderCallbacks<Object>{
    @Override
    public void onActivityCreated(Bundle savedInstanceState) {
    //See if the loader is there
    Loader<Object> loader = getActivity().getSupportLoaderManager().getLoader(LOADER_ARTICLE_SUMMARY);
    if (loader != null && !loader.isReset()) {
    //reset it as data needs to be loaded again because activity was created again
    getActivity().getSupportLoaderManager().restartLoader(LOADER_ARTICLE_SUMMARY, args, this);
    } else {
    //instantiate the loader
    getActivity().getSupportLoaderManager().initLoader(LOADER_ARTICLE_SUMMARY, args, this);
    }

    }
    @Override
    public Loader<Object> onCreateLoader(int id, Bundle args) {
    //instantiate the loader here depending upon loader id there could be several loaders that you might instantiate
    if(id==LOADER_ARTICLE_SUMMARY){
    String rawQuery="select _id , "+ FeedSQLLiteHelper.COLUMN_ARTICLE_TITLE+ " , "+
    FeedSQLLiteHelper.COLUMN_ARTICLE_READ+ " , "+ FeedSQLLiteHelper.COLUMN_ARTICLE_IS_FAVORITE
    +" , "+FeedSQLLiteHelper.COLUMN_ARTICLE_DOWNLOADED+ " , "+ FeedSQLLiteHelper.COLUMN_ARTICLE_SUMMARY
    +" , "+FeedSQLLiteHelper.COLUMN_ARTICLE_FULL_CONTENT+" , "+FeedSQLLiteHelper.COLUMN_ARTICLE_CREATED_DATE
    +" , "+FeedSQLLiteHelper.COLUMN_ARTICLE_URL +" from "+FeedSQLLiteHelper.TABLE_ARTICLES + " where _id = ?";
    String articleId= args.getString(FeedSQLLiteHelper.COLUMN_ID);
    Loader loader=new SQLiteCursorLoader(this.getActivity().getApplicationContext(),fdb.getDbHelper(), rawQuery, new String []{articleId});
    return loader;
    }
    else {
    Loader loader= new ArticleLoader(getActivity().getApplicationContext(),id,args);
    return loader;
    }
    }
    @Override
    public void onLoadFinished(Loader<Object> loader, Object data) {
    /** Here you are supplied with the data, so you could update the cursor of the adapter
    Just make sure you check the loader id before updating the adapter
    */
    int id=loader.getId();
    if(id==LOADER_ARTICLE_SUMMARY){
    Cursor cursor=(Cursor)data;
    if(!cursor.isClosed()){
    //do pojo conversion
    article=cursorToFeedArticle(cursor);
    loadDataIntoWebView(article);

    }
    }
    else{
    Map<String,String> result=(Map)data;
    article.setArticleContent(result.get("ARTICLE_FULL_CONTENT"));
    article.setArticleDownloaded(result.get("ARTICLE_DOWNLOADED"));
    loadDataIntoWebView(article);
    }
    }
    @Override
    public void onLoaderReset(Loader<Object> arg0) {
    /** remove references to the data of the loader if its a cursor just dereference it
    * In this case since we do not have a cursor leave it empty.
    */


    }

    }



  3. Adapter:  This is used to hold the data to view binding. In other words, this object holds data which can be accessed by the view and the data to view binding information which tells how that data is mapped to the  view components of the view.


  4. Activity: An activity is like a page. It represents a view which might be composed of several child views (fragments) or just hold its own view.


  5. Fragment:  It is a child view, it has its own layout, but it exists only within an activity. As a fragment can be reused within different activities, so it should not hold any activity specific references.  You can attach the fragment to an activity at runtime using FragmentManager API’s , but if the fragment is listed in the activity’s layout then it cannot be removed. So, generally it is better to use the FragmentManager API’s then to have this restriction.




    articleFragment = (ArticleFragment) getSupportFragmentManager()
    .findFragmentByTag(ArticleFragment.tag);

    if (articleFragment == null) {
    //create a new instance of the fragment
    articleFragment = new ArticleFragment();
    Bundle args = new Bundle();
    args.putString("KEY","VALUE");
    articleFragment.setArguments(args);
    //replace the framelayout with the article fragment's content
    getSupportFragmentManager()
    .beginTransaction()
    .replace(R.id.content_frame_article_fragment,
    articleFragment, ArticleFragment.tag).commit();
    }

    else {
    // create a method onUpdateFromActivity, just supply it with new arguments
    articleFragment.onUpdateFromActivity(args);
    }



    Also Fragment instances can be retained, but in that case its lifecycle is different. It is better to have them retained because obviously it speeds up your application.


  6. Intent and Intent Filter: Intent is basically an object which is used to invoke another activity, a service instance, etc. You generally supply it with an action when invoking IntentService instance.   IntentFilter on the other hand is used to filter out what actions can a particular service, activity or a fragment can handle.


  7. IntentService and BroadcastReceiver:  IntentService listens for intent actions and maintains a queue of requests which are handled one after the other sequentially. It runs on a separate background thread, so one can perform synchronous tasks on it.  You must implement its onHandleIntent method to perform the tasks required.  Once the task is completed, you should broadcast an intent signifying so and in your activity or fragment listen for the intent using BroadcastReceiver. 


    //1. start an intentservice
    Intent feedPullServiceIntent = new Intent(this.getActivity().getApplicationContext(), PullFeedsService.class);
    feedPullServiceIntent.setAction(ACTION_UPDATE_FEEDS);
    feedPullServiceIntent.putExtra("feedName", feedName);
    getActivity().getApplicationContext().startService(feedPullServiceIntent);


    public class PullFeedsService extends IntentService {

    //2. handle the action
    @Override
    protected void onHandleIntent(Intent intent) {
    if(action==ACTION_UPDATE_FEEDS){
    if(isConnected()){
    //fetch and parse the feeds
    .....

    //3. notify completion
    notifyFetchComplete(ACTION_FEED_FETCH_COMPLETE,feedName);
    }

    }
    if(action==DELETE_FEEDS){
    //delete the feeds in the batch
    notifyFetchComplete(ACTION_DELETE_COMPLETE,feedName);
    }
    }
    private void notifyFetchComplete(String action, String feedName) {
    Intent i = new Intent();
    i.setAction(action);
    Bundle result=new Bundle();
    if(action==ACTION_FEED_FETCH_COMPLETE){
    Bundle result=new Bundle();
    result.putString("KEY", "VALUE");
    }
    else if(action==ACTION_DELETE_COMPLETE){
    Bundle result=new Bundle();
    result.putString("KEY", "VALUE");
    }
    i.putExtras(result);
    //4. Brodcast the event
    LocalBroadcastManager.getInstance(getApplicationContext()).sendBroadcast(i);
    }

    //in your fragment or activity

    private BroadcastReceiver receiver = new BroadcastReceiver() {
    //5. called only when the fragment receives the specific intents as specificied in the intent filter while registering
    @Override
    public void onReceive(Context context, Intent intent) {

    if(intent.getAction()==PullFeedsService.ACTION_FEED_FETCH_COMPLETE){
    if(refreshItem!=null){
    refreshItem.getActionView().clearAnimation();
    refreshItem.setActionView(null);
    }
    Bundle result=intent.getExtras();
    String refreshForFeed=result.getString("RefreshForFeed");
    if(refreshForFeed.equals("ALL")||refreshForFeed.equals(feedName)){
    //reset loaders
    resetLoaders();
    }


    }

    }
    };

    @Override
    public void onPause() {
    ...
    //fragment paused so unregister the receiver
    LocalBroadcastManager.getInstance(getActivity().getApplicationContext()).unregisterReceiver(receiver);

    }

    @Override
    public void onResume() {
    ...
    //0. fragment running so register the receiver to listen for events
    //specify a particular intent filter
    IntentFilter intentFilter = new IntentFilter();
    intentFilter.addAction(PullFeedsService.ACTION_FEED_FETCH_COMPLETE);
    intentFilter.addAction(PullFeedsService.ACTION_NOT_CONNECTED);
    LocalBroadcastManager.getInstance(this.getActivity().getApplicationContext()).registerReceiver(receiver, intentFilter);

    }



That’s it for now. In future, I might expound upon these individual elements,.

Posted on Friday, October 03, 2014 by Unknown

Just an update : these days, I am developing a RSS reader app on android. I am using android framework for development and not  Oracle MAF because it tends to be free of license restrictions, plus it performs better. I am not particularly fond of android’s ContentProvider API’s , as I find them to be inflexible, so I am going to take the conventional approach of programming.  In the coming posts, I am going to share tips and tricks on android application development  and of course  when the application is developed(which will take some time) it will be available on play store for free with its source posted on GitHub.

Posted on Friday, October 03, 2014 by Unknown

Jun 12, 2014

In this post i will share the security utility for performing operations on IAM Fortress applications using it’s java API’s. Use this utility only if you plan on having custom code to manage Fortress as fortress already provides a J2ee application to manage its policies. The code can be used coupled with earlier examples to integrate fortress as a security provider for your ADF essentials application. Also, you can make task flows for performing user, role, permission management using this utility.  The code is mentioned below

Posted on Thursday, June 12, 2014 by Unknown

May 23, 2014

Recently a user contacted me with the requirement that he needed to assign enterprise groups to application roles dynamically. Normally, people can manage this through enterprise manager, but there might have been a specific requirement for that user.  I have already written various posts on ADF security which cover utility methods that use OPSS API’s to perform operations on the enterprise OID, OpenLdap etc.  In this post i will cover this specific requirement; although the given code is very crude, it is there only to serve as an example and the process is what matters.

The code is shown below.

package model;

import oracle.security.jps.service.policystore.ApplicationPolicy;
import oracle.security.jps.service.policystore.PolicyStore;
import java.security.Principal;

import java.util.Hashtable;

import oracle.adf.share.logging.ADFLogger;

import oracle.security.idm.IMException;
import oracle.security.idm.IdentityStore;
import oracle.security.idm.IdentityStoreFactory;
import oracle.security.idm.IdentityStoreFactoryBuilder;
import oracle.security.idm.Role;
import oracle.security.idm.RoleManager;
import oracle.security.idm.User;
import oracle.security.idm.UserManager;
import oracle.security.idm.providers.oid.OIDIdentityStoreFactory;
import oracle.security.jps.JpsContext;
import oracle.security.jps.JpsContextFactory;
import oracle.security.jps.JpsException;
import oracle.security.jps.internal.idstore.ldap.LdapIdentityStore;
import oracle.security.jps.service.idstore.IdentityStoreException;
import oracle.security.jps.service.idstore.IdentityStoreService;
import oracle.security.jps.service.policystore.ApplicationPolicy;
import oracle.security.jps.service.policystore.PolicyObjectNotFoundException;
import oracle.security.jps.service.policystore.PolicyStore;
import oracle.security.jps.service.policystore.PolicyStoreException;
import oracle.security.jps.service.policystore.info.common.InvalidArgumentException;

public class DemoJpsMethods {
public static final ADFLogger DemoJpsLogger=ADFLogger.createADFLogger(DemoJpsMethods.class);
public static final String DEV_GROUP="DevGroup";
public static final String TRAINING_GROUP="TrainingGroup";
public static final String DEV_APP_ROLE="DevAppRole";
public static final String TRAINING_APP_ROLE="TrainingAppRole";
public DemoJpsMethods() {
super();
}

/**
* Dummy method just for testing so not parameterized
*/
public void addUserToAppGroup(){
JpsContext ctxt = IdentityStoreConfigurator.jpsCtxt;
LdapIdentityStore idstoreService =
(LdapIdentityStore)ctxt.getServiceInstance(IdentityStoreService.class);
IdentityStore idmStore=null;
try {
idmStore = idstoreService.getIdmStore();

} catch (IdentityStoreException e) {
throw new RuntimeException(e);
}
User tom=null;
User helen=null;
UserManager um=null;
RoleManager rm=null;
Role trainingRole=null;
Role devRole=null;

try {
um= idmStore.getUserManager();
rm=idmStore.getRoleManager();
} catch (IMException e) {
try {
idmStore.close();
} catch (IMException f) {
}
throw new RuntimeException(e);
}
try {
trainingRole =
idmStore.searchRole(IdentityStore.SEARCH_BY_NAME, TRAINING_GROUP);
devRole =
idmStore.searchRole(IdentityStore.SEARCH_BY_NAME, DEV_GROUP);
} catch (IMException e) {
DemoJpsLogger.severe("Rethrow exception because roles ARE NOT found");
try {
idmStore.close();
} catch (IMException f) {
}
throw new RuntimeException(e);
}


try {
tom = idmStore.searchUser("tom");
helen = idmStore.searchUser("helen");


//if it comes till here user is found

} catch (IMException e) {
DemoJpsLogger.severe("Ignore exception because user is likely not found");
}
//User not found create ?
if(tom==null){
try {
tom= um.createUser("tom", "Welcome_1".toCharArray());

} catch (IMException e) {
try {
idmStore.close();
} catch (IMException f) {
}
throw new RuntimeException(e);

}

}
//User not found create ?
if(helen==null){
try {
helen=um.createUser("helen", "Welcome_1".toCharArray());

} catch (IMException e) {
try {
idmStore.close();
} catch (IMException f) {
}
throw new RuntimeException(e);
}

}
try {
rm.grantRole(trainingRole,tom.getPrincipal());
rm.grantRole(devRole,helen.getPrincipal());
} catch (IMException e) {
try {
idmStore.close();
} catch (IMException f) {
}
throw new RuntimeException(e);
}

if(idmStore!=null){
try {
idmStore.close();
} catch (IMException f) {
}
}



}
public void addAppGroupToAppRole() {
JpsContext ctxt = IdentityStoreConfigurator.jpsCtxt;
LdapIdentityStore idstoreService =
(LdapIdentityStore)ctxt.getServiceInstance(IdentityStoreService.class);
IdentityStore idmStore=null;
try {
idmStore = idstoreService.getIdmStore();
} catch (IdentityStoreException e) {
throw new RuntimeException(e);
}
PolicyStore ps = ctxt.getServiceInstance(PolicyStore.class);
ApplicationPolicy policy;
Role trainingRole=null;
Role devRole=null;
try {
trainingRole =
idmStore.searchRole(IdentityStore.SEARCH_BY_NAME, TRAINING_GROUP);
devRole =
idmStore.searchRole(IdentityStore.SEARCH_BY_NAME, DEV_GROUP);
} catch (IMException e) {
try {
idmStore.close();
} catch (IMException f) {
}
throw new RuntimeException(e);
}



try {
policy = ps.getApplicationPolicy("DemoAppSecurity#V2.0");
policy.addPrincipalToAppRole(trainingRole.getPrincipal(), TRAINING_APP_ROLE);
policy.addPrincipalToAppRole(devRole.getPrincipal(), DEV_APP_ROLE);
} catch (PolicyStoreException e) {
throw new RuntimeException(e);
} catch (IMException e) {
throw new RuntimeException(e);
}

if(idmStore!=null){
try {
idmStore.close();
} catch (IMException f) {
}
}

}



/**
* This nested private class is used for configuring and initializing the context factory
* @author Ramandeep Nanda
*/
private static final class IdentityStoreConfigurator {
private static final JpsContext jpsCtxt=initializeFactory();


private static JpsContext initializeFactory(){
String methodName = Thread.currentThread().getStackTrace()[1].getMethodName();
JpsContextFactory tempFactory;
JpsContext jpsContext;
try {
tempFactory=JpsContextFactory.getContextFactory();
jpsContext=tempFactory.getContext();
}
catch (JpsException e) {
DemoJpsLogger.severe("Exception in "+methodName + " " +e.getMessage() +" ", e);
throw new RuntimeException("Exception in "+methodName + " " +e.getMessage() +" ", e);
}
return jpsContext;
}



}
}


 



So basically there are two methods, the first one just creates the user and assigns them to the enterprise groups, the second one is the one we’re interested in because this one first accesses the application policy and then dynamically assigns the enterprise group to the application role .



We are not done just yet, there are a few steps you need to configure which are mentioned below.




  • Assign PolicyStoreAccessPermission to weblogic user in system-jazn-data.xml.
    <grant>
    <grantee>
    <principals>
    <principal>
    <class>weblogic.security.principal.WLSUserImpl</class>
    <name>weblogic</name>
    </principal>
    </principals>
    </grantee>
    <permissions>
    <permission>
    <class>oracle.security.jps.service.policystore.PolicyStoreAccessPermission</class>
    <name>context=SYSTEM</name>
    <actions>*</actions>
    </permission>
    </permissions>
    </grant>



  • Although i have added the application stripe grant in the application jazn-data.xml, you need to be aware of that. It is mentioned below:
      <grant>
    <grantee>
    <principals>
    <principal>
    <class>oracle.security.jps.internal.core.principals.JpsXmlUserImpl</class>
    <name>weblogic</name>
    </principal>
    </principals>
    </grantee>
    <permissions>
    <permission>
    <class>oracle.security.jps.service.policystore.PolicyStoreAccessPermission</class>
    <name>context=APPLICATION,name=DemoAppSecurity#V2.0</name>
    <actions>*</actions>
    </permission>
    </permissions>
    </grant>


     



  • Run the application and log in with weblogic user i.e login.html page


  • There will be two buttons one creates and adds a user to a group, second adds enterprise group to app role


  • For first, You need an ldap browser to view the changes, use the embedded ldap connection details.


  • For second, you need just to view the system-jazn-data.xml file, here you will see enterprise group added to app role.



The application can be downloaded from here. Application Link

Posted on Friday, May 23, 2014 by Unknown

The in built primefaces charting components can be extended like any other faces component. In this post i will explain how to extend the in built pie chart component to use custom renderer which in turn will use NVD3 pie chart component. On a similar line you can similarly create your own custom faces component also.

The code is mentioned below.

package com.blogspot.ramannanda.ui.chart;

import java.io.IOException;
import java.util.Iterator;

import javax.faces.component.UIComponent;
import javax.faces.context.FacesContext;
import javax.faces.context.ResponseWriter;

import org.primefaces.component.chart.UIChart;
import org.primefaces.component.chart.line.LineChart;
import org.primefaces.component.chart.pie.PieChart;
import org.primefaces.component.chart.pie.PieChartRenderer;
import org.primefaces.model.chart.PieChartModel;

public class NVD3PieChartRenderer extends PieChartRenderer {
@Override
public void encodeEnd(FacesContext context, UIComponent component) throws IOException{
PieChart chart = (PieChart) component;

encodeMarkup(context, chart);
encodeScript(context, chart);
}
@Override
protected void encodeScript(FacesContext context, UIChart uichart) throws IOException{
ResponseWriter writer = context.getResponseWriter();
PieChart chart = (PieChart) uichart;
String clientId = chart.getClientId(context);
String d3Id=chart.getClientId(context);
d3Id="pieChart"+d3Id.substring(d3Id.lastIndexOf(":")+1, d3Id.length());
startScript(writer, clientId);
encodeData(context,chart);
writer.write("nv.addGraph(function() {"+
"var chart =nv.models.pieChart()"+
".x(function(d) { return d.key })"+
".y(function(d) { return d.value })"+
".color(d3.scale.category20().range())"
+ ".margin({top: 30, right: 30, bottom: 30, left: 30});");
writer.write("d3.select('#"+d3Id+" svg')"+
".datum(mydata"+d3Id+")"+
".call(chart);");
writer.write("nv.utils.windowResize(chart.update);");
writer.write("return chart; });");
endScript(writer);

}
@Override
protected void encodeData(FacesContext context, PieChart chart) throws IOException {
ResponseWriter writer = context.getResponseWriter();
String d3Id=chart.getClientId(context);
d3Id="pieChart"+d3Id.substring(d3Id.lastIndexOf(":")+1, d3Id.length());
writer.write("var mydata"+d3Id+"=[" );
PieChartModel model = (PieChartModel) chart.getValue();

for(Iterator<String> it = model.getData().keySet().iterator(); it.hasNext();) {
String key = it.next();
Number value = model.getData().get(key);
if(value==null){
value=0;
}
writer.write("{key:'" + escapeText(key) + "',value:'" + value + "'}");

if(it.hasNext())
writer.write(",");
}

writer.write("];");

}
@Override
protected void encodeMarkup(FacesContext context, UIChart chart)
throws IOException {
ResponseWriter writer = context.getResponseWriter();
String d3Id=chart.getClientId(context);

writer.startElement("div", null);
writer.writeAttribute("id", "pieChart"+d3Id.substring(d3Id.lastIndexOf(":")+1, d3Id.length()), null);
if(chart.getStyle() != null)
writer.writeAttribute("style", chart.getStyle(), "style");
writer.writeAttribute("class", "chart1", "styleClass");
writer.writeAttribute("align", "center", "align");
writer.startElement("h3",null);
writer.writeText(chart.getTitle(),null);
writer.endElement("h3");
writer.startElement("svg",null);
writer.endElement("svg");

writer.endElement("div");

}

}


 



If you had followed the previous intro to d3.js and you are aware with JSF, this code sample should be easy to follow. Basically we create a div element with the client id and then svg element under it, we then do a selection of the svg element under the div. Here the thing to note is that we are using datum function rather than data function, whereas the data function computes a join, datum function just binds the data to the existing element and does not compute the join.



To register the renderer add the following into faces configuration file.



  <renderer>
<component-family>org.primefaces.component</component-family>
<renderer-type>org.primefaces.component.chart.PieChartRenderer</renderer-type>
<renderer-class>com.blogspot.ramannanda.ui.chart.NVD3PieChartRenderer</renderer-class>
</renderer>


Also make sure to include the following files as they are part of nvd3 and d3 libraries into your *.xhtml page.



<link href="../src/nv.d3.css" rel="stylesheet" type="text/css">
<script src="../lib/d3.v3.js"></script>
<script src="../nv.d3.js"></script>

<script src="../src/models/legend.js"></script>

<script src="../src/models/pie.js"></script>

<script src="../src/models/pieChart.js"></script>

<script src="../src/utils.js"></script>
<script>
<style type="text/css">

body {
overflow-y:scroll;
overflow-x:scroll;
}

text {
font: 12px sans-serif;
}

svg {
display: block;
}

.chart1{
padding:10px;
min-width: 150px;
min-height: 150px;


}
</style>


And this is how the pie chart component now looks. You can select which data to show and the state of the pie chart changes to whatever data is selected along with a nice transition effect. Cool uh ?.



piechar1



piechart2

Posted on Friday, May 23, 2014 by Unknown

D3.js is a framework for data driven documents. You can use it to create data visualizations. If you require some charting component for your java application you can easily create components based on this library or a derivative of it. In this post i will highlight some of its features via a html example and in the next post will give you an example of how you can change the renderer of prime faces component to use nvd3 component( A set of charting components built using  d3.js).

D3 has a syntax similar to jquery that means you can easily select, append, remove existing elements, chain method calls etc. I will first present the source code of the example and then explain it step by step.

<html>
<head>
<style>
.chart {
fill:steelblue;
}

.bar {
fill:steelblue;
}
.chart text {
fill: black;
font: 10px sans-serif;
text-anchor: middle;
}
.axis text {
font: 10px sans-serif;
}

.axis path,
.axis line {
fill: none;
stroke: #000;
shape-rendering: crispEdges;
}



div.tooltip {
position: absolute;
text-align: center;
width: 80px;
height: 40px;
padding: 8px;
font: 10px sans-serif;
background: #ddd;
border: solid 1px #aaa;
border-radius: 8px;
pointer-events: none;
}
</style>
<script src="d3/d3.js" charset="utf-8"></script>

</head>
<body>

<svg class="chart"></svg>

</body>
<script>
//Data to be used
var data = [{key:'Mathematics',value:80}, {key:'English',value:70},{key:'Physics',value:90},{key:'Biology',value:60},{key:'History',value:80},{key:'Chemistry',value:50},{key:'Geology',value:50}];

//specify margins
var margin = {top: 20, right: 30, bottom: 30, left: 40},
width = 600 - margin.left - margin.right,
height = 500 - margin.top - margin.bottom;
//calculate each barwidth dynamically
var barWidth=width/data.length;
//Since we do not have integer coordinates and only string labels the x axis scale is ordinal
//here we have specified only range and domain will be defined later
var x = d3.scale.ordinal()
.rangeRoundBands([0, width], .4);
//Since cordinates start at top so we want the higher values to have lower y
var y = d3.scale.linear()
.range([height, 0]);
//Define an x axis, specify the scale to use and orientation
var xAxis = d3.svg.axis()
.scale(x)
.orient("bottom");
//Define a Y axis
var yAxis = d3.svg.axis()
.scale(y)
.orient("left");
// Selecting an element to create the initial g element under svg and translate the element to (x,y) of margins

var chart=d3.select(".chart").attr("width",width+margin.left + margin.right).attr("height",height+ margin.top + margin.bottom).append("g").attr("transform", "translate("+margin.left+","+margin.top+")");
//Now we specify the domains here x domain is special because we use map function to map each key and hence construct a domain for x axis
x.domain(data.map(function(d) { return d.key; }));
//y domain is simply from 0 to the maximum value of data, here we used data with a function to calculate the maximum of associative array
y.domain([0, d3.max(data, function(d) { return d.value;})]);

//Now Use enter selection to Append rectangle, notice y is given the value of hieght so that the bars are at base, add showtop function to //show tooltips
chart.selectAll(".bar").data(data).enter().append("rect").attr("class","bar").attr("x",function(d,i){return x(d.key);}).
attr("y",height).attr("width",x.rangeBand()).attr("height",0).on("mousemove",showTip);
//append the tooltip element and set its opacity to invisible value

var div = d3.select("body").append("div")
.attr("class", "tooltip")
.style("opacity", 1e-6);
//Call x and yaxis functions and append them to the chart div element
chart.append("g").attr("class","x axis").attr("transform", "translate(0,"+height+")").call(xAxis);
chart.append("g")
.attr("class", "y axis")
.call(yAxis);
//This is where we set the actual values of the y positions and give the bars the transition effect
chart.selectAll(".bar").transition().duration("1000").attr("y",function(d,i){return y(d.value);}).attr("height",function(d,i){ return height-y(d.value);});
//give a olive transition effect on mouseover
chart.selectAll(".bar").on("mouseover",function (){d3.select(this).transition().duration("1000").style("fill","olive");mouseoverTip();
});

function showTip(d) {
div.text("Scored "+d.value+" in "+ d.key)
.style("left", (d3.event.pageX - 34) + "px")
.style("top", (d3.event.pageY - 12) + "px");
}
//onmouseout transition
chart.selectAll(".bar").on("mouseout",function (){
d3.select(this).transition().duration("1000").style("fill","steelblue");
mouseoutTip();
}
);



function mouseoutTip() {
div.transition()
.duration(500)
.style("opacity", 1e-6);
}
function mouseoverTip() {
div.transition()
.duration(500)
.style("opacity", 1);
}

//Append label to y axis

chart.append("g")
.attr("class", "y axis")
.call(yAxis)
.append("text")
.attr("transform", "rotate(-90)")
.attr("y", 6)
.attr("dy", ".71em")
.style("text-anchor", "end")
.text("Subject Scores");



</script>
</html>


I have added explanation comments to the source code. Now I will explain the details of each section.




  • D3 Selections(Enter/Update/Exit):  In D3 you generally work with selections which binds your data to elements or element. You can think of them as joins. Also the data that is joined is available to all the functions in the join statement, this is important because you can then perform functions on the data.

    • Enter Selection : In case of enter selection when data is bound to elements all the non existing elements for which is there is not data are created. So let’s say you have initially no elements and want to create them, then you do a Enter Selection.  In this example enter selection is used to append rectangle elements and since no existing elements with class .bar exist, new ones will be created.


    • Update Selection: This is used to update the elements  with new  attribute values; The thing to note is that the elements must exist, only then will they be updated with new data values.


    • Exit Selection: This is the opposite of enter selection, here all the elements for which there is no data are removed.




  • Dynamic chart scaling: You generally define a scaling function for representing data on graph by dynamically mapping data values to their respective positions in the graph. The scaling function can be linear, ordinal etc. In the above mentioned example we do not have x coordinates but only String labels, so to map the labels to the corresponding positions, we use ordinal scale. The domain function accepts an array of values, So to generate the array of values from the data that we use javascript’s map function. The range function in case of x coordinates is rangeroundbands which divides the entire range into n bands(1. It gives exact integer values 2. Here n is the number of values in the domain 3. It accepts second argument which specifies spacing between elements). For y scale we use a linear scale because we want our values to scale linearly.


  • Coordinate System: If you are aware about element positioning, you will know that the top left corner represents the origin i.e x=0,y=0 position. So for us to draw the bars at the base, we give higher values of y in range to lower values in the domain, this is so because we have to subtract these values from height. So y becomes higher for lower values but height of the bar becomes lower.


  • Transitions: In this example we use two different types of transitions. 1) Position transition: If you followed the example you would have noticed that initial y position of the rect svg element is set to height, this is done because we want the bars to transition from base to their respective heights. Also transition requires that the elements must exist before it can occur, this is why it is done in the second step. 2) Style transition: If you hover your mouse over the bars, you would notice that the bar’s colour  changes to olive smoothly, this is done by changing the fill value of rect elements and using the transition function.


  • Events: The selected elements can easily be bound to events using the on() function. In this example transition and tooltip functions are bound to events such as mouseover, mouseout, mousemove etc.



The screenshot of the graph is given below. In the next post i will cover an example of how to extend a primefaces component to use nvd3 library.



 



d3example

Posted on Friday, May 23, 2014 by Unknown

Jan 21, 2014

I was recently given a requirement to drive the entity attribute validations at runtime rather than at design time, with the following criteria.
  1. Ability to change fields to required at runtime.
  2. To be able to define validation message for field at runtime
  3. Define validations such as field must be numeric, or contain only alphabets (Used Regular expression for this)
  4. Localization of validation messages
To accomplish this we can use database dictionary views along with custom database tables to store the validations and perform validations at runtime. The data model diagram is shown below.
DEMO_RUNTIME_VALThe key thing then is to use database dictionary views along with these database tables to validate attributes of the entity for a pattern (regular expression).

The class that contains the method to perform validations is CustomEntityImpl and each entity that one wants to enable for runtime validation must extend this class and call its validateCustomEntity method in its entity method validator.

    protected Boolean validateCustomEntity(){
          
            ResourceBundleDef rbd=getEntityDef().getResourceBundleDef();
            PropertiesBundleDef pbd=null;
            if(rbd==null){
            pbd = new PropertiesBundleDef(this.getEntityDef());
            pbd.setPropertiesFile("com.blogspot.ramannanda.demos.validationapp.model.ModelBundle");
            rbd=pbd;
            }
            ArrayList<RowValException> exceptions=new ArrayList<RowValException>();
            EntityImpl entObj= this;
                       String tableName=this.getEntityDef().getSource();
                       tableName=tableName.substring(tableName.lastIndexOf(".")+1,tableName.length());
                       AttributeDef[] attrDefs= this.getEntityDef().getAttributeDefs();
                       
                       ValidationsVOImpl vo=(ValidationsVOImpl) this.getDBTransaction().getRootApplicationModule().findViewObject("ValidationsVO12");
                       ViewCriteriaManager vcm=vo.getViewCriteriaManager();
                       ViewCriteria vc=vcm.getViewCriteria("ValidationsVOCriteria");
                       vo.setbTableName(tableName);
                       vo.applyViewCriteria(vc, false);
                       vo.executeQuery();
                       RowSetIterator it=vo.createRowSetIterator(null);
                       ArrayList<AttrValException> attrValExceptions=new ArrayList<AttrValException>();
                       while(it.hasNext()){
                          ValidationsVORowImpl row=(ValidationsVORowImpl)it.next();
                          String pattern=row.getValidationPattern();
                          String fieldName=row.getAttribName();
                           
                               for(int i=0;i<attrDefs.length;i++){
                                    String attrName=attrDefs[i].getName();
                                    String columnName=attrDefs[i].getColumnName();
                                   if(columnName.equals(fieldName)){
                                       String attribRequired=row.getAttribRequired();
                                       Object attribValue=entObj.getAttribute(attrName);
                                      
                                       ArrayList<JboException> list=new ArrayList<JboException>(2);
                                       if(attribRequired.equalsIgnoreCase("Y"))
                                       {
                                           if(attribValue==null || ((String)attribValue).isEmpty()){
                                          String errorMessageKey=row.getRequiredMsgKey();
                                           if(errorMessageKey!=null&&!errorMessageKey.isEmpty()){
                                           JboException exception=new JboException(getMessageForKey(errorMessageKey),null,new Object[]{attrName});
                                                   list.add(exception);
                                               }
                                           else{
                                                   AttrSetValException exception=
                                                       new AttrSetValException(AttrValException.TYP_DEF_ENTITY_OBJECT,  rbd,
                                                      "DEFAULT_REQUIRED_ERROR", entObj.getStructureDef().getFullName(),
                                                       attrName,attribValue, null);
                                                       list.add(exception);
                                               }
                                            
                                           }
                                       }
                                       if(((attribValue!=null)&& !((String)attribValue).isEmpty())){
                                       boolean result=validatePattern(pattern,(String)entObj.getAttribute(attrName));
                                       if(!result){
                                          
                                               String errorMessageKey=row.getValidationMsgKey();
                                               if(errorMessageKey!=null&&!errorMessageKey.isEmpty()){
                                                       JboException exception=new JboException(getMessageForKey(errorMessageKey),null,new Object[]{attrName,attribValue});
                                                               list.add(exception);
                                                       list.add(exception);
                                                   }
                                               else{
                                                       AttrSetValException exception=
                                                           new AttrSetValException(AttrValException.TYP_DEF_ENTITY_OBJECT,  rbd,
                                                          "DEFAULT_INCORRECT_VAL_ERROR", entObj.getStructureDef().getFullName(),
                                                           attrName,attribValue, null);
                                                           list.add(exception);
                                                   }
                                           }
                                       }
                                       if(list.size()>0){
                                               AttrValException ave =
                                                        new AttrValException(CSMessageBundle.class, CSMessageBundle.EXC_VAL_VR_VALIDATE_FAILED, entObj.getStructureDef().getFullName(),
                                                                             attrName,attribValue, list, false);
                                               attrValExceptions.add(ave);
                                           }
                                       
                                       
                                   } 
                           }
                      
                   }
                       it.closeRowSetIterator();
                       if(attrValExceptions.size()>0){
                           RowValException rowValException= new RowValException(CSMessageBundle.class, CSMessageBundle.EXC_VAL_VR_VALIDATE_FAILED,
                                entObj.getStructureDef().getFullName(), entObj.getKey(), attrValExceptions);
                           exceptions.add(rowValException);
                           }

                 
     
                   if(exceptions.size()>0){
                       throw new
                        RowValException(CSMessageBundle.class, CSMessageBundle.EXC_VAL_VR_VALIDATE_FAILED,
                                                           getStructureDef().getFullName(), getKey(), exceptions);
                       }
            return true;
        
        }
   

    /**
     * This method returns false or true depending upon whether the field is valid or not
     * @param patternString
     * @param value
     */
    private boolean validatePattern(String patternString, String value) {
        Pattern pattern =Pattern.compile(patternString);
        Matcher matcher= pattern.matcher(value);
        return matcher.matches();
        
    }
    

    
    private String getMessageForKey(String msgKey){
            Locale defaultLocale=Locale.US;
            String defaultMessage=null;
            String localMessage=null;
            Locale locale = this.getDBTransaction().getSession().
              getLocale();
             String localString=locale.toLanguageTag();
            ValidationMessagesVOImpl vo=(ValidationMessagesVOImpl) this.getDBTransaction().getRootApplicationModule().findViewObject("ValidationMessagesVO12");
            ViewCriteriaManager vcm=vo.getViewCriteriaManager();
            ViewCriteria vc=vcm.getViewCriteria("ValidationMessagesVOCriteria1");
            vo.setbMessageKey(msgKey);
            vo.applyViewCriteria(vc, false);
            vo.executeQuery();
            RowSetIterator it=vo.createRowSetIterator(null);
            while(it.hasNext()){
                ValidationMessagesVORowImpl row=(ValidationMessagesVORowImpl) it.next();
                if(row.getMessageLocale().equals(defaultLocale.toLanguageTag())){
                      defaultMessage=row.getMessageText();
                    }
                if(row.getMessageLocale().equals(localString)){
                        localMessage=row.getMessageText();
                        break;
                    }
                }
            it.closeRowSetIterator();
            if(localMessage==null){
                return defaultMessage;
                }
            return localMessage;
        }





In the sample application i have enabled a required and pattern constraint as shown in the following screenshots for dummy entity object’s attribute.





runtimevalidation_defineruntimevalidation_define_messagesruntimevalidation_define_1


runtimevalidationerror   





The application source and sql script can be downloaded from the below mentioned link.


Source Code





Note: If you want to test the application you must create tables under user “raman”.





Edit: To skip validation for existing attribute that has not been modified you can use the following snippet.


                                       Object attribValue=getAttribute(attrName);
                                       int attribIndex=getAttributeIndexOf(attrName);
                                       Object oldAttribValue=getPostedAttribute(attribIndex);
                                       if(attribValue==null || !attribValue.equals(oldAttribValue)){
                                       ArrayList<JboException> list=new ArrayList<JboException>(2);
                                       if(attribRequired.equalsIgnoreCase("Y"))
                                       {
                                       ....................

Posted on Tuesday, January 21, 2014 by Unknown