Apache NIFI provides various options to retry/wait in processors. If you want to implement custom processor with out of the box nifii solutions to wait on certain condition/external-resource then it would be complex workflow.
Objective: Identify the regions in India where crime rate is emerging?
Pellucid tutorial using java modules, REPL(READ EVALUATE Print Loop) jshell and how to launch jshell inline command from shell
Jdk-9 is one of the major shift for java developers, it has so many powerful tools and jshell is one of them. Jshell shall be used to run small commands, utilities, expressions, tough it helps to avoid developing many standalone utilities but it shouldn’t be used to run java classes directly.
Spark processing multiline csv EOLs in text column
Below code snippet shows how to save RDD output input single file with header:
Spark Socket streaming example windows:
Windows OS doesn’t provide any netcat utility and if you are trying to test your spark streaming socket program in windows then either you download external netcat utility or create socket program equivalent to netcat.
I am gonna demonstrate step by step setup of spark project in this post and explore few basics Spark dataset operations in Java.
In this post I will show you how to create Apache Beam Spark Runner project using Maven.
Java to json schema maven example:
YAML and JSON are simple and nice format for structured data and easier for human to read and write than XML. But there have been no schema for YAML such as RelaxNG or DTD. Below is an example that convert java object to json schema. Check post for java object to yaml examplejava to yaml
Java to yaml schema maven example:
YAML and JSON are simple and nice format for structured data and easier for human to read and write than XML. But there have been no schema for YAML such as RelaxNG or DTD. Below is an example that convert java object to yaml.
Debugging custom libraries hive update logging to console.
When launch Hive cli change logging set root logging or your library logging to DEBUG or INFO and print to console-
Hive hadoop jar files conflicts for custom UDF/Serde
Hive NoClassDefFoundError error auxiliary path issue is very common. Sometimes even you add jar into classpath using below hive command, hive throws NoClassDefFound error-
Solrcloud 4 inconsistent results count-commitwithinms issue:
Solr Multithreaded concurrent atomic updates problem:
Solr has few limitations for the data ingestion, as it doesn’t provide row level lock over document.
I face this problem while uploading data in bulk to solr5 in multithread environment and I solved it by solrj client side lock.
When concurrent threads try to make atomic update on a multivalued field of a document at the same time, few threads changes get overridden and it happens because last thread update take sometime to get indexed.
HBase Manager design example
Getting Started with Raspberry Pi 2 mod B is a small guide that answers all your basic queries
Raspberry pi or its latest version pi 2 mod B is a (I would not say cheap) low cost, credit card sized computer. Pi can plug into your computer monitor or TV set and uses a standard keyboard and mouse (wired or Wireless). It is capable of delivering everything that is expected from a desktop computer, from programming, gaming, internet browsing, playing HD videos and a lot more.
It is now easy to handle asynchronous request in loadrunner. I am not going to start with differentiating synchronous and asynchronous request here, as I suppose if you are here you already know the difference.
Lanuch Hbase Mapreduce Job
Add all the required jars to a variable separated by(,)
in my case I want to supply all hbase jars to job
Get Hbase Columns with Column Families
Hbase doesn’t provide any client API to get all the column qualifiers. If your table has millions of rows and you need to get all the qualifiers then it takes very long time to get all columns by standalone program. Here is, map reduce job that will help you to get all columns with family.
Prepare the Job Driver:
Below is the LoadRunner and Internet Explorer compatibility matrix -
Sometimes it is confusing, when we observe the difference in Avg, max and min values of response time in “Summary report” and values in “Average transaction response time” graph.
If you are experiencing the usual SSL protocol error during the replay of script in VUgen. Try the below function with SSL version as “TLS” to resolve the “SSL protocol error when attempting to read with host”
Pagination with Hbase
Hbase tips and tricks
Extract XSD from WSDL in Java..
Hbase mulitple operations with bash script or run the bulk of command from bash script
HBase latest version shell commands that provide a jruby-style object-oriented references for tables and reference variable can be used to perform the hbase operation directly in ruby shell, but there are no way to run the same into bash script.
Using bash pipe a table operation(put,get, scan…) can be spawn to hbase shell, single operation will be performed at a time and variable table referencing can’t be implemented with bash scripting.
Here we will run the table operations such as puts, scans, and admin functionality such as disabling, dropping, describing tables using bash script.
A single hbase operation can be launched from bash like this-
Below program can provide sum of any number of matrices of any degree.
In this part I am going to answer below questions regarding zombie process
Configure and install Hue-
1- Hue Native lib dependencies-
Hue has many modules which are dependent on native library.
Hive Query Over Hbase-
Generate indexing using map reduce can’t be done in distributed mode of mapreduce as each line number is sequential and unique. To achieve this I tried to generate index using single reducer job.
LoadRunner Correlationis one of the most important concept in loadrunner, especially when you are working on web protocols in loadrunner like web(http/html), sap web or Oracle web applications. When dealing with complex scripts, we encounter scenarios where the usual web_reg_save_param function is not able to solve the purpose.
There are 2 types of capture levels - socket level and wininet level data in loadrunner. These 2 capture levels are available in recording options. They specify the tool, where to hook and capture the communication packets from.
Sometime, LoadRunner captures all events during recording but throws an Error during script generation - “Error during code generation. The Vuser script was not generated.”
There are lot of common errors that we keep encountering while using Loadrunner, like HTTP status code 401, 500, 403 etc.
Saving a file to the local machine during recording of a pdf download script in loadrunner VUgen, by clicking on”save as” button, is a client side activity and does not get recorded. But the action on web page which results the file download box gets recorded in VUgen and actually downloads the file (pdf file/Zip file).
There is an inbuilt function in loadrunner that can be used to capture data from string by specifying delimiters. strtok function can be used to do the trick -
LoadRunner Correlation is one of the most important concept in loadrunner, especially when you are working on web protocols in loadrunner like web(http/html), sap web or Oracle web applications etc.
The “error -26601: Decompression function wgzMemDecompressBuffer failed, return code=-5 (Z_BUF_ERROR), inSize=0, inUse=0, outUse=0” in loadrunner occurs because of insufficient buffer size set in the runtime settings. The Network buffer size is set to 12288 bytes, by default.
The following performance counters for .net applications, can act as general guidelines for different performance problems, related to the .net application under test.
The Error “Server has shutdown the connection prematurely” is often seen during performance test execution in controller.
Scripting a file Upload scenario is usually pretty straightforward, just do few correlations and the script runs without any errors.
How to detect a memory leak
The basic symptoms of memory bottlenecks that affect application performance are -
Below areas can be looked into for performance tuning an application:-
Step by step guide to find performance bottlenecks in Linux system:-