- java 23
- LoadRunner 14
- bigdata 11
- hbase 7
- spark 7
- LoadRunner Scripting Tricks 6
- hadoop 6
- Loadrunner Errors 5
- hive 4
- UNIX 3
- Internet of Things 2
- Loadrunner Correlation 2
- Raspberry Pi 2
- memory leak 2
- solr 2
- LoadRunner Analysis 1
- Performance bottlenecks 1
- Zombie 1
- apache Beam 1
- awk 1
- data-visualization 1
- graphql 1
- map-reduce 1
- matrices 1
- nifi 1
- performance monitors and counters 1
java
Apache NIFI Retry & wait in custom processor
Apache NIFI provides various options to retry/wait in processors. If you want to implement custom processor with out of the box nifii solutions to wait on certain condition/external-resource then it would be complex workflow.
GraphQL SPQR- how to access the native ServletRequest and Response
Apache Spark Operations implementation in Java
Introduction to jshell/REPL and how to launch inline command in jshell from shell
Pellucid tutorial using java modules, REPL(READ EVALUATE Print Loop) jshell and how to launch jshell inline command from shell
Jdk-9 is one of the major shift for java developers, it has so many powerful tools and jshell is one of them. Jshell shall be used to run small commands, utilities, expressions, tough it helps to avoid developing many standalone utilities but it shouldn’t be used to run java classes directly.
Spark solution for multiline csv which has EOLs in text column
Spark processing multiline csv EOLs in text column
how to save Spark RDD output in single file with header using java
Below code snippet shows how to save RDD output input single file with header:
Spark Socket streaming example windows
Spark Socket streaming example windows:
Windows OS doesn’t provide any netcat utility and if you are trying to test your spark streaming socket program in windows then either you download external netcat utility or create socket program equivalent to netcat.
Spark Dataset Operations in java
I am gonna demonstrate step by step setup of spark project in this post and explore few basics Spark dataset operations in Java.
Mapr Certified Spark Developer (MCSD) guide
Apache Beam Spark Runner example using Maven
In this post I will show you how to create Apache Beam Spark Runner project using Maven.
Java to json schema maven example
Java to json schema maven example:
YAML and JSON are simple and nice format for structured data and easier for human to read and write than XML. But there have been no schema for YAML such as RelaxNG or DTD. Below is an example that convert java object to json schema. Check post for java object to yaml examplejava to yaml
Java to yaml schema maven example
Java to yaml schema maven example:
YAML and JSON are simple and nice format for structured data and easier for human to read and write than XML. But there have been no schema for YAML such as RelaxNG or DTD. Below is an example that convert java object to yaml.
Debugging custom libraries hive update logging to console
Debugging custom libraries hive update logging to console.
When launch Hive cli change logging set root logging or your library logging to DEBUG or INFO and print to console-
hive hadoop jar files conflicts for custom UDF/Serde
Hive hadoop jar files conflicts for custom UDF/Serde
Hive NoClassDefFoundError auxiliary path issue
Hive NoClassDefFoundError error auxiliary path issue is very common. Sometimes even you add jar into classpath using below hive command, hive throws NoClassDefFound error-
Solrcloud 4 inconsistent results count-commitwithinms issue
Solrcloud 4 inconsistent results count-commitwithinms issue:
Solr Multithreaded concurrent atomic updates problem.
Solr Multithreaded concurrent atomic updates problem:
Solr has few limitations for the data ingestion, as it doesn’t provide row level lock over document.
I face this problem while uploading data in bulk to solr5 in multithread environment and I solved it by solrj client side lock.
When concurrent threads try to make atomic update on a multivalued field of a document at the same time, few threads changes get overridden and it happens because last thread update take sometime to get indexed.
HBase Manager design example
HBase Manager design example
Lanuch Hbase Mapreduce Job
Lanuch Hbase Mapreduce Job
Add all the required jars to a variable separated by(,)
like jar=a,b
in my case I want to supply all hbase jars to job
Get Hbase Columns with Column Families
Get Hbase Columns with Column Families
Hbase doesn’t provide any client API to get all the column qualifiers. If your table has millions of rows and you need to get all the qualifiers then it takes very long time to get all columns by standalone program. Here is, map reduce job that will help you to get all columns with family.
Prepare the Job Driver:
Pagination with Hbase
Pagination with Hbase
Extract XSD from WSDL in Java
Extract XSD from WSDL in Java..
Matrices Sum Map Reducer
Matrices Sum Map Reducer
Matrix sum is the operation of adding matrices by adding corresponding entries together.
Entrywise sum
The sum of two m × n (pronounced “m by n”) matrices A and B, denoted by A + B, is again an m × n matrix computed by adding corresponding elements
LoadRunner
Handling Asynchronous request in LoadRunner
It is now easy to handle asynchronous request in loadrunner. I am not going to start with differentiating synchronous and asynchronous request here, as I suppose if you are here you already know the difference.
LoadRunner and Internet Explorer compatibility matrix
Below is the LoadRunner and Internet Explorer compatibility matrix -
why there is difference between values of average response time in Summary Report and Average transaction response time graph
Sometimes it is confusing, when we observe the difference in Avg, max and min values of response time in “Summary report” and values in “Average transaction response time” graph.
SSL protocol error when attempting to read with host
If you are experiencing the usual SSL protocol error during the replay of script in VUgen. Try the below function with SSL version as “TLS” to resolve the “SSL protocol error when attempting to read with host”
Loadrunner Correlation of dynamic boundary
LoadRunner Correlationis one of the most important concept in loadrunner, especially when you are working on web protocols in loadrunner like web(http/html), sap web or Oracle web applications. When dealing with complex scripts, we encounter scenarios where the usual web_reg_save_param function is not able to solve the purpose.
Difference between Socket level and Wininet level data in Loadrunner
There are 2 types of capture levels - socket level and wininet level data in loadrunner. These 2 capture levels are available in recording options. They specify the tool, where to hook and capture the communication packets from.
“Error during code generation. The Vuser script was not generated” - Loadrunner
Sometime, LoadRunner captures all events during recording but throws an Error during script generation - “Error during code generation. The Vuser script was not generated.”
Error HTTP Status Code 401 Unauthorized during script replay
There are lot of common errors that we keep encountering while using Loadrunner, like HTTP status code 401, 500, 403 etc.
How to create and validate a pdf download script in LoadRunner
Saving a file to the local machine during recording of a pdf download script in loadrunner VUgen, by clicking on”save as” button, is a client side activity and does not get recorded. But the action on web page which results the file download box gets recorded in VUgen and actually downloads the file (pdf file/Zip file).
strtok - Capture a sub string or data from a string based on delimiters in Loadrunner
There is an inbuilt function in loadrunner that can be used to capture data from string by specifying delimiters. strtok function can be used to do the trick -
LoadRunner Correlation
LoadRunner Correlation is one of the most important concept in loadrunner, especially when you are working on web protocols in loadrunner like web(http/html), sap web or Oracle web applications etc.
Loadrunner : Error -26601: Decompression function wgzMemDecompressBuffer failed, return code=-5 (Z_BUF_ERROR), inSize=0, inUse=0, outUse=0
The “error -26601: Decompression function wgzMemDecompressBuffer failed, return code=-5 (Z_BUF_ERROR), inSize=0, inUse=0, outUse=0” in loadrunner occurs because of insufficient buffer size set in the runtime settings. The Network buffer size is set to 12288 bytes, by default.
LoadRunner Controller Error :- Server has shutdown the connection prematurely
The Error “Server has shutdown the connection prematurely” is often seen during performance test execution in controller.
How to resolve LoadRunner 500 internal server error while testing an upload file scenario
Scripting a file Upload scenario is usually pretty straightforward, just do few correlations and the script runs without any errors.
bigdata
Apache NIFI Retry & wait in custom processor
Apache NIFI provides various options to retry/wait in processors. If you want to implement custom processor with out of the box nifii solutions to wait on certain condition/external-resource then it would be complex workflow.
Apache Spark Operations implementation in Java
Spark solution for multiline csv which has EOLs in text column
Spark processing multiline csv EOLs in text column
Debugging custom libraries hive update logging to console
Debugging custom libraries hive update logging to console.
When launch Hive cli change logging set root logging or your library logging to DEBUG or INFO and print to console-
hive hadoop jar files conflicts for custom UDF/Serde
Hive hadoop jar files conflicts for custom UDF/Serde
Hive NoClassDefFoundError auxiliary path issue
Hive NoClassDefFoundError error auxiliary path issue is very common. Sometimes even you add jar into classpath using below hive command, hive throws NoClassDefFound error-
Solrcloud 4 inconsistent results count-commitwithinms issue
Solrcloud 4 inconsistent results count-commitwithinms issue:
Solr Multithreaded concurrent atomic updates problem.
Solr Multithreaded concurrent atomic updates problem:
Solr has few limitations for the data ingestion, as it doesn’t provide row level lock over document.
I face this problem while uploading data in bulk to solr5 in multithread environment and I solved it by solrj client side lock.
When concurrent threads try to make atomic update on a multivalued field of a document at the same time, few threads changes get overridden and it happens because last thread update take sometime to get indexed.
Pagination with Hbase
Pagination with Hbase
Hbase tips and tricks
Hbase tips and tricks
Configure and install Hue
Configure and install Hue-
1- Hue Native lib dependencies-
Hue has many modules which are dependent on native library.
hbase
HBase Manager design example
HBase Manager design example
Lanuch Hbase Mapreduce Job
Lanuch Hbase Mapreduce Job
Add all the required jars to a variable separated by(,)
like jar=a,b
in my case I want to supply all hbase jars to job
Get Hbase Columns with Column Families
Get Hbase Columns with Column Families
Hbase doesn’t provide any client API to get all the column qualifiers. If your table has millions of rows and you need to get all the qualifiers then it takes very long time to get all columns by standalone program. Here is, map reduce job that will help you to get all columns with family.
Prepare the Job Driver:
Pagination with Hbase
Pagination with Hbase
Hbase tips and tricks
Hbase tips and tricks
Hbase mulitple operations with bash script
Hbase mulitple operations with bash script or run the bulk of command from bash script
HBase latest version shell commands that provide a jruby-style object-oriented references for tables and reference variable can be used to perform the hbase operation directly in ruby shell, but there are no way to run the same into bash script.
Using bash pipe a table operation(put,get, scan…) can be spawn to hbase shell, single operation will be performed at a time and variable table referencing can’t be implemented with bash scripting.
Here we will run the table operations such as puts, scans, and admin functionality such as disabling, dropping, describing tables using bash script.
A single hbase operation can be launched from bash like this-
Hive Query Over Hbase
Hive Query Over Hbase-
spark
Apache Spark Operations implementation in Java
Spark solution for multiline csv which has EOLs in text column
Spark processing multiline csv EOLs in text column
how to save Spark RDD output in single file with header using java
Below code snippet shows how to save RDD output input single file with header:
Spark Socket streaming example windows
Spark Socket streaming example windows:
Windows OS doesn’t provide any netcat utility and if you are trying to test your spark streaming socket program in windows then either you download external netcat utility or create socket program equivalent to netcat.
Spark Dataset Operations in java
I am gonna demonstrate step by step setup of spark project in this post and explore few basics Spark dataset operations in Java.
Mapr Certified Spark Developer (MCSD) guide
Apache Beam Spark Runner example using Maven
In this post I will show you how to create Apache Beam Spark Runner project using Maven.
LoadRunner Scripting Tricks
Handling Asynchronous request in LoadRunner
It is now easy to handle asynchronous request in loadrunner. I am not going to start with differentiating synchronous and asynchronous request here, as I suppose if you are here you already know the difference.
Loadrunner Correlation of dynamic boundary
LoadRunner Correlationis one of the most important concept in loadrunner, especially when you are working on web protocols in loadrunner like web(http/html), sap web or Oracle web applications. When dealing with complex scripts, we encounter scenarios where the usual web_reg_save_param function is not able to solve the purpose.
Difference between Socket level and Wininet level data in Loadrunner
There are 2 types of capture levels - socket level and wininet level data in loadrunner. These 2 capture levels are available in recording options. They specify the tool, where to hook and capture the communication packets from.
“Error during code generation. The Vuser script was not generated” - Loadrunner
Sometime, LoadRunner captures all events during recording but throws an Error during script generation - “Error during code generation. The Vuser script was not generated.”
How to create and validate a pdf download script in LoadRunner
Saving a file to the local machine during recording of a pdf download script in loadrunner VUgen, by clicking on”save as” button, is a client side activity and does not get recorded. But the action on web page which results the file download box gets recorded in VUgen and actually downloads the file (pdf file/Zip file).
strtok - Capture a sub string or data from a string based on delimiters in Loadrunner
There is an inbuilt function in loadrunner that can be used to capture data from string by specifying delimiters. strtok function can be used to do the trick -
hadoop
Apache Beam Spark Runner example using Maven
In this post I will show you how to create Apache Beam Spark Runner project using Maven.
Lanuch Hbase Mapreduce Job
Lanuch Hbase Mapreduce Job
Add all the required jars to a variable separated by(,)
like jar=a,b
in my case I want to supply all hbase jars to job
Get Hbase Columns with Column Families
Get Hbase Columns with Column Families
Hbase doesn’t provide any client API to get all the column qualifiers. If your table has millions of rows and you need to get all the qualifiers then it takes very long time to get all columns by standalone program. Here is, map reduce job that will help you to get all columns with family.
Prepare the Job Driver:
Hbase mulitple operations with bash script
Hbase mulitple operations with bash script or run the bulk of command from bash script
HBase latest version shell commands that provide a jruby-style object-oriented references for tables and reference variable can be used to perform the hbase operation directly in ruby shell, but there are no way to run the same into bash script.
Using bash pipe a table operation(put,get, scan…) can be spawn to hbase shell, single operation will be performed at a time and variable table referencing can’t be implemented with bash scripting.
Here we will run the table operations such as puts, scans, and admin functionality such as disabling, dropping, describing tables using bash script.
A single hbase operation can be launched from bash like this-
Matrices Sum Map Reducer
Matrices Sum Map Reducer
Matrix sum is the operation of adding matrices by adding corresponding entries together.
Entrywise sum
The sum of two m × n (pronounced “m by n”) matrices A and B, denoted by A + B, is again an m × n matrix computed by adding corresponding elements
Indexing Using Map Reduce
Generate indexing using map reduce can’t be done in distributed mode of mapreduce as each line number is sequential and unique. To achieve this I tried to generate index using single reducer job.
Loadrunner Errors
SSL protocol error when attempting to read with host
If you are experiencing the usual SSL protocol error during the replay of script in VUgen. Try the below function with SSL version as “TLS” to resolve the “SSL protocol error when attempting to read with host”
Error HTTP Status Code 401 Unauthorized during script replay
There are lot of common errors that we keep encountering while using Loadrunner, like HTTP status code 401, 500, 403 etc.
Loadrunner : Error -26601: Decompression function wgzMemDecompressBuffer failed, return code=-5 (Z_BUF_ERROR), inSize=0, inUse=0, outUse=0
The “error -26601: Decompression function wgzMemDecompressBuffer failed, return code=-5 (Z_BUF_ERROR), inSize=0, inUse=0, outUse=0” in loadrunner occurs because of insufficient buffer size set in the runtime settings. The Network buffer size is set to 12288 bytes, by default.
LoadRunner Controller Error :- Server has shutdown the connection prematurely
The Error “Server has shutdown the connection prematurely” is often seen during performance test execution in controller.
How to resolve LoadRunner 500 internal server error while testing an upload file scenario
Scripting a file Upload scenario is usually pretty straightforward, just do few correlations and the script runs without any errors.
hive
Debugging custom libraries hive update logging to console
Debugging custom libraries hive update logging to console.
When launch Hive cli change logging set root logging or your library logging to DEBUG or INFO and print to console-
hive hadoop jar files conflicts for custom UDF/Serde
Hive hadoop jar files conflicts for custom UDF/Serde
Hive NoClassDefFoundError auxiliary path issue
Hive NoClassDefFoundError error auxiliary path issue is very common. Sometimes even you add jar into classpath using below hive command, hive throws NoClassDefFound error-
Hive Query Over Hbase
Hive Query Over Hbase-
UNIX
Hbase mulitple operations with bash script
Hbase mulitple operations with bash script or run the bulk of command from bash script
HBase latest version shell commands that provide a jruby-style object-oriented references for tables and reference variable can be used to perform the hbase operation directly in ruby shell, but there are no way to run the same into bash script.
Using bash pipe a table operation(put,get, scan…) can be spawn to hbase shell, single operation will be performed at a time and variable table referencing can’t be implemented with bash scripting.
Here we will run the table operations such as puts, scans, and admin functionality such as disabling, dropping, describing tables using bash script.
A single hbase operation can be launched from bash like this-
Matrices Sum using awk
Below program can provide sum of any number of matrices of any degree.
UNIX: Zombie processes
In this part I am going to answer below questions regarding zombie process
Internet of Things
Getting Started with Raspberry Pi 2 mod B
Getting Started with Raspberry Pi 2 mod B is a small guide that answers all your basic queries
A brief introduction of Raspberry pi 2 mod B
Raspberry pi or its latest version pi 2 mod B is a (I would not say cheap) low cost, credit card sized computer. Pi can plug into your computer monitor or TV set and uses a standard keyboard and mouse (wired or Wireless). It is capable of delivering everything that is expected from a desktop computer, from programming, gaming, internet browsing, playing HD videos and a lot more.
Loadrunner Correlation
Loadrunner Correlation of dynamic boundary
LoadRunner Correlationis one of the most important concept in loadrunner, especially when you are working on web protocols in loadrunner like web(http/html), sap web or Oracle web applications. When dealing with complex scripts, we encounter scenarios where the usual web_reg_save_param function is not able to solve the purpose.
LoadRunner Correlation
LoadRunner Correlation is one of the most important concept in loadrunner, especially when you are working on web protocols in loadrunner like web(http/html), sap web or Oracle web applications etc.
Raspberry Pi
Getting Started with Raspberry Pi 2 mod B
Getting Started with Raspberry Pi 2 mod B is a small guide that answers all your basic queries
A brief introduction of Raspberry pi 2 mod B
Raspberry pi or its latest version pi 2 mod B is a (I would not say cheap) low cost, credit card sized computer. Pi can plug into your computer monitor or TV set and uses a standard keyboard and mouse (wired or Wireless). It is capable of delivering everything that is expected from a desktop computer, from programming, gaming, internet browsing, playing HD videos and a lot more.
memory leak
How to detect a memory leak
How to detect a memory leak
How to Pinpoint Memory bottlenecks
The basic symptoms of memory bottlenecks that affect application performance are -
solr
Solrcloud 4 inconsistent results count-commitwithinms issue
Solrcloud 4 inconsistent results count-commitwithinms issue:
Solr Multithreaded concurrent atomic updates problem.
Solr Multithreaded concurrent atomic updates problem:
Solr has few limitations for the data ingestion, as it doesn’t provide row level lock over document.
I face this problem while uploading data in bulk to solr5 in multithread environment and I solved it by solrj client side lock.
When concurrent threads try to make atomic update on a multivalued field of a document at the same time, few threads changes get overridden and it happens because last thread update take sometime to get indexed.
LoadRunner Analysis
why there is difference between values of average response time in Summary Report and Average transaction response time graph
Sometimes it is confusing, when we observe the difference in Avg, max and min values of response time in “Summary report” and values in “Average transaction response time” graph.
Performance bottlenecks
How to Pinpoint Memory bottlenecks
The basic symptoms of memory bottlenecks that affect application performance are -
Zombie
UNIX: Zombie processes
In this part I am going to answer below questions regarding zombie process
apache Beam
Apache Beam Spark Runner example using Maven
In this post I will show you how to create Apache Beam Spark Runner project using Maven.
awk
Matrices Sum using awk
Below program can provide sum of any number of matrices of any degree.
data-visualization
India IPC crimes Analysis
Objective: Identify the regions in India where crime rate is emerging?
graphql
Back to Top ↑map-reduce
Indexing Using Map Reduce
Generate indexing using map reduce can’t be done in distributed mode of mapreduce as each line number is sequential and unique. To achieve this I tried to generate index using single reducer job.
matrices
Matrices Sum using awk
Below program can provide sum of any number of matrices of any degree.
nifi
Apache NIFI Retry & wait in custom processor
Apache NIFI provides various options to retry/wait in processors. If you want to implement custom processor with out of the box nifii solutions to wait on certain condition/external-resource then it would be complex workflow.
performance monitors and counters
Important performance counters for .net applications
The following performance counters for .net applications, can act as general guidelines for different performance problems, related to the .net application under test.