Improving Performance of Data Science Applications in Python

Objectives: The objective of this paper is to focus on the need of performance optimization of Data Science applications with an aim to enhance the speed, responsiveness, and effective resource utilization. Methods: This paper identifies the factors which effect the performance of an application and then classifies them into internal and external factors. This study mainly focuses on internal factors such as execution speed and space requirement. It uses code developed in Python as use cases to explore and examine features which improve the performance and effectiveness of Data Science (DS) applications. Findings: This paper provides comprehensive way of improving performance of applications by implementing features of Python like generator, vectorization, profiling, concurrency & parallelism, and caching. It is observed that space requirement reduces with the use of generators in code, leading to better space management. The conventional way of writing code in Python required additional 5.2 MiB (Mebibyte) space, while there was no additional space requirement with the use of generators in the same code. It is further observed that vectorization feature of Python helps in reduc