Tag Archive

java

Introduction to jshell/REPL and how to launch inline command in jshell from shell

Pellucid tutorial using java modules, REPL(READ EVALUATE Print Loop) jshell and how to launch jshell inline command from shell
Jdk-9 is one of the major shift for java developers, it has so many powerful tools and jshell is one of them. Jshell shall be used to run small commands, utilities, expressions, tough it helps to avoid developing many standalone utilities but it shouldn’t be used to run java classes directly.

Spark Socket streaming example windows

Spark Socket streaming example windows:
Windows OS doesn’t provide any netcat utility and if you are trying to test your spark streaming socket program in windows then either you download external netcat utility or create socket program equivalent to netcat.

Spark Dataset Operations in java

I am gonna demonstrate step by step setup of spark project in this post and explore few basics Spark dataset operations in Java.

Java to json schema maven example

Java to json schema maven example:
YAML and JSON are simple and nice format for structured data and easier for human to read and write than XML. But there have been no schema for YAML such as RelaxNG or DTD. Below is an example that convert java object to json schema. Check post for java object to yaml examplejava to yaml

Java to yaml schema maven example

Java to yaml schema maven example:
YAML and JSON are simple and nice format for structured data and easier for human to read and write than XML. But there have been no schema for YAML such as RelaxNG or DTD. Below is an example that convert java object to yaml.

Hive NoClassDefFoundError auxiliary path issue

Hive NoClassDefFoundError error auxiliary path issue is very common. Sometimes even you add jar into classpath using below hive command, hive throws NoClassDefFound error-

Solr Multithreaded concurrent atomic updates problem.

Solr Multithreaded concurrent atomic updates problem:
Solr has few limitations for the data ingestion, as it doesn’t provide row level lock over document.
I face this problem while uploading data in bulk to solr5 in multithread environment and I solved it by solrj client side lock.
When concurrent threads try to make atomic update on a multivalued field of a document at the same time, few threads changes get overridden and it happens because last thread update take sometime to get indexed.

Lanuch Hbase Mapreduce Job

Lanuch Hbase Mapreduce Job
Add all the required jars to a variable separated by(,)
like jar=a,b
in my case I want to supply all hbase jars to job

Get Hbase Columns with Column Families

Get Hbase Columns with Column Families
Hbase doesn’t provide any client API to get all the column qualifiers. If your table has millions of rows and you need to get all the qualifiers then it takes very long time to get all columns by standalone program. Here is, map reduce job that will help you to get all columns with family.
Prepare the Job Driver:

Matrices Sum Map Reducer

Matrices Sum Map Reducer
Matrix sum is the operation of adding matrices by adding corresponding entries together.
Entrywise sum
The sum of two m × n (pronounced “m by n”) matrices A and B, denoted by A + B, is again an m × n matrix computed by adding corresponding elements
matraix sum

Back to Top ↑

LoadRunner

Handling Asynchronous request in LoadRunner

It is now easy to handle asynchronous request in loadrunner. I am not going to start with differentiating synchronous and asynchronous request here, as I suppose if you are here you already know the difference.

SSL protocol error when attempting to read with host

If you are experiencing the usual SSL protocol error during the replay of script in VUgen. Try the below function with SSL version as “TLS” to resolve the “SSL protocol error when attempting to read with host”

Loadrunner Correlation of dynamic boundary

LoadRunner Correlationis one of the most important concept in loadrunner, especially when you are working on web protocols in loadrunner like web(http/html), sap web or Oracle web applications. When dealing with complex scripts, we encounter scenarios where the usual web_reg_save_param function is not able to solve the purpose.

How to create and validate a pdf download script in LoadRunner

Saving a file to the local machine during recording of a pdf download script in loadrunner VUgen, by clicking on”save as” button, is a client side activity and does not get recorded. But the action on web page which results the file download box gets recorded in VUgen and actually downloads the file (pdf file/Zip file).

LoadRunner Correlation

LoadRunner Correlation is one of the most important concept in loadrunner, especially when you are working on web protocols in loadrunner like web(http/html), sap web or Oracle web applications etc.

Back to Top ↑

bigdata

Hive NoClassDefFoundError auxiliary path issue

Hive NoClassDefFoundError error auxiliary path issue is very common. Sometimes even you add jar into classpath using below hive command, hive throws NoClassDefFound error-

Solr Multithreaded concurrent atomic updates problem.

Solr Multithreaded concurrent atomic updates problem:
Solr has few limitations for the data ingestion, as it doesn’t provide row level lock over document.
I face this problem while uploading data in bulk to solr5 in multithread environment and I solved it by solrj client side lock.
When concurrent threads try to make atomic update on a multivalued field of a document at the same time, few threads changes get overridden and it happens because last thread update take sometime to get indexed.

Configure and install Hue

Configure and install Hue-
1- Hue Native lib dependencies-
Hue has many modules which are dependent on native library.

Back to Top ↑

hbase

Lanuch Hbase Mapreduce Job

Lanuch Hbase Mapreduce Job
Add all the required jars to a variable separated by(,)
like jar=a,b
in my case I want to supply all hbase jars to job

Get Hbase Columns with Column Families

Get Hbase Columns with Column Families
Hbase doesn’t provide any client API to get all the column qualifiers. If your table has millions of rows and you need to get all the qualifiers then it takes very long time to get all columns by standalone program. Here is, map reduce job that will help you to get all columns with family.
Prepare the Job Driver:

Hbase mulitple operations with bash script

Hbase mulitple operations with bash script or run the bulk of command from bash script
HBase latest version shell commands that provide a jruby-style object-oriented references for tables and reference variable can be used to perform the hbase operation directly in ruby shell, but there are no way to run the same into bash script.
Using bash pipe a table operation(put,get, scan…) can be spawn to hbase shell, single operation will be performed at a time and variable table referencing can’t be implemented with bash scripting.
Here we will run the table operations such as puts, scans, and admin functionality such as disabling, dropping, describing tables using bash script.
A single hbase operation can be launched from bash like this-

Back to Top ↑

spark

Spark Socket streaming example windows

Spark Socket streaming example windows:
Windows OS doesn’t provide any netcat utility and if you are trying to test your spark streaming socket program in windows then either you download external netcat utility or create socket program equivalent to netcat.

Spark Dataset Operations in java

I am gonna demonstrate step by step setup of spark project in this post and explore few basics Spark dataset operations in Java.

Back to Top ↑

LoadRunner Scripting Tricks

Handling Asynchronous request in LoadRunner

It is now easy to handle asynchronous request in loadrunner. I am not going to start with differentiating synchronous and asynchronous request here, as I suppose if you are here you already know the difference.

Loadrunner Correlation of dynamic boundary

LoadRunner Correlationis one of the most important concept in loadrunner, especially when you are working on web protocols in loadrunner like web(http/html), sap web or Oracle web applications. When dealing with complex scripts, we encounter scenarios where the usual web_reg_save_param function is not able to solve the purpose.

How to create and validate a pdf download script in LoadRunner

Saving a file to the local machine during recording of a pdf download script in loadrunner VUgen, by clicking on”save as” button, is a client side activity and does not get recorded. But the action on web page which results the file download box gets recorded in VUgen and actually downloads the file (pdf file/Zip file).

Back to Top ↑

hadoop

Lanuch Hbase Mapreduce Job

Lanuch Hbase Mapreduce Job
Add all the required jars to a variable separated by(,)
like jar=a,b
in my case I want to supply all hbase jars to job

Get Hbase Columns with Column Families

Get Hbase Columns with Column Families
Hbase doesn’t provide any client API to get all the column qualifiers. If your table has millions of rows and you need to get all the qualifiers then it takes very long time to get all columns by standalone program. Here is, map reduce job that will help you to get all columns with family.
Prepare the Job Driver:

Hbase mulitple operations with bash script

Hbase mulitple operations with bash script or run the bulk of command from bash script
HBase latest version shell commands that provide a jruby-style object-oriented references for tables and reference variable can be used to perform the hbase operation directly in ruby shell, but there are no way to run the same into bash script.
Using bash pipe a table operation(put,get, scan…) can be spawn to hbase shell, single operation will be performed at a time and variable table referencing can’t be implemented with bash scripting.
Here we will run the table operations such as puts, scans, and admin functionality such as disabling, dropping, describing tables using bash script.
A single hbase operation can be launched from bash like this-

Matrices Sum Map Reducer

Matrices Sum Map Reducer
Matrix sum is the operation of adding matrices by adding corresponding entries together.
Entrywise sum
The sum of two m × n (pronounced “m by n”) matrices A and B, denoted by A + B, is again an m × n matrix computed by adding corresponding elements
matraix sum

Indexing Using Map Reduce

Generate indexing using map reduce can’t be done in distributed mode of  mapreduce as each line number is sequential and unique. To achieve this I tried to generate index using single reducer job.

Back to Top ↑

Loadrunner Errors

SSL protocol error when attempting to read with host

If you are experiencing the usual SSL protocol error during the replay of script in VUgen. Try the below function with SSL version as “TLS” to resolve the “SSL protocol error when attempting to read with host”

Back to Top ↑

hive

Hive NoClassDefFoundError auxiliary path issue

Hive NoClassDefFoundError error auxiliary path issue is very common. Sometimes even you add jar into classpath using below hive command, hive throws NoClassDefFound error-

Back to Top ↑

UNIX

Hbase mulitple operations with bash script

Hbase mulitple operations with bash script or run the bulk of command from bash script
HBase latest version shell commands that provide a jruby-style object-oriented references for tables and reference variable can be used to perform the hbase operation directly in ruby shell, but there are no way to run the same into bash script.
Using bash pipe a table operation(put,get, scan…) can be spawn to hbase shell, single operation will be performed at a time and variable table referencing can’t be implemented with bash scripting.
Here we will run the table operations such as puts, scans, and admin functionality such as disabling, dropping, describing tables using bash script.
A single hbase operation can be launched from bash like this-

UNIX: Zombie processes

In this part I am going to answer below questions regarding zombie process

Back to Top ↑

Internet of Things

A brief introduction of Raspberry pi 2 mod B

Raspberry pi or its latest version pi 2 mod B is a (I would not say cheap) low cost, credit card sized computer. Pi can plug into your computer monitor or TV set and uses a standard keyboard and mouse (wired or Wireless). It is capable of delivering everything that is expected from a desktop computer, from programming, gaming, internet browsing, playing HD videos and a lot more.

Back to Top ↑

Loadrunner Correlation

Loadrunner Correlation of dynamic boundary

LoadRunner Correlationis one of the most important concept in loadrunner, especially when you are working on web protocols in loadrunner like web(http/html), sap web or Oracle web applications. When dealing with complex scripts, we encounter scenarios where the usual web_reg_save_param function is not able to solve the purpose.

LoadRunner Correlation

LoadRunner Correlation is one of the most important concept in loadrunner, especially when you are working on web protocols in loadrunner like web(http/html), sap web or Oracle web applications etc.

Back to Top ↑

Raspberry Pi

A brief introduction of Raspberry pi 2 mod B

Raspberry pi or its latest version pi 2 mod B is a (I would not say cheap) low cost, credit card sized computer. Pi can plug into your computer monitor or TV set and uses a standard keyboard and mouse (wired or Wireless). It is capable of delivering everything that is expected from a desktop computer, from programming, gaming, internet browsing, playing HD videos and a lot more.

Back to Top ↑

memory leak

Back to Top ↑

solr

Solr Multithreaded concurrent atomic updates problem.

Solr Multithreaded concurrent atomic updates problem:
Solr has few limitations for the data ingestion, as it doesn’t provide row level lock over document.
I face this problem while uploading data in bulk to solr5 in multithread environment and I solved it by solrj client side lock.
When concurrent threads try to make atomic update on a multivalued field of a document at the same time, few threads changes get overridden and it happens because last thread update take sometime to get indexed.

Back to Top ↑

LoadRunner Analysis

Back to Top ↑

Performance bottlenecks

Back to Top ↑

Zombie

UNIX: Zombie processes

In this part I am going to answer below questions regarding zombie process

Back to Top ↑

apache Beam

Back to Top ↑

awk

Back to Top ↑

data-visualization

Back to Top ↑

graphql

Back to Top ↑

map-reduce

Indexing Using Map Reduce

Generate indexing using map reduce can’t be done in distributed mode of  mapreduce as each line number is sequential and unique. To achieve this I tried to generate index using single reducer job.

Back to Top ↑

matrices

Back to Top ↑

performance monitors and counters

Back to Top ↑