1. Introduction

This paper is an extension of work originally presented in 19th International Conference on Computer and Information Technology (ICCIT). IEEE, 2016 [1]. The number of users using the internet is rapidly rising with the advancement of the internet as it helps users to find information of their interest with in very short possible time. Compared to the number of internet users in 1995 which was only 1% of world’s total population, it has increased up to 40% in recent years. This mass communication media has gained so much popularity that people have been using this on daily basis.

The internet serves its users through web applications and websites. The reasons why web applications have gained so much popularity are given below-

• Zero installation cost
• Reduce business costs
• Reach anybody, anywhere in the world
• 24/7 accessibility
• Access to the latest information

2.       Background

Web applications are being used in medical, banking transactions, online buying selling, online marketing, communication etc. Web application can help doctors to prescribe elder people [2]. With the escalation in number of web applications, the amount of information is increasing. Any kind of web applications have numerous number of contents within it. With large number of contents it is often difficult to satisfy the users. Most of the time users have to go through a large number of information to find out their required information which is leading towards usability problems. To evaluate and improve web applications, usability issue has become an important field of study. Nowadays Facebook, YouTube etc. have become so much popular because of their advanced recommendation systems. In order to solve the usability problem researchers are consistently working on how user experience can be improved. How the user’s experience can be improved is described in [3]. This paper focuses on how web application contents can be made more user friendly through mouse tracking. We propose a generalized approach that will solve any kind of web application’s usability problems at the same time improving the user experience.

      A large number of contents often make it hard for the users to find their items of interest. Solution to this problem is to make user-centered web applications. Developing a user centered web application is not an easy task as it’s often very difficult to take user’s feedback manually. There have already been many works for knowing user’s preferences and expectations automatically without bothering the user [4]. An earlier study [5] shows adaptive navigation mechanisms and illustrates how these can help web applications. The paper [6] focuses on architecture, database design, local concepts, and user interface design for developing content management system. User interest can be predicted by eye tracking [7]. An alternative way for predicting user’s engagement can be mouse activity and cursor movement. Mouse activity can be used for web designing to make the user interface more precise for user usage [8]. In our previous work [1] it’s described how user-modeling and recommendation by mouse-tracking can be used for the construction of an intelligent e-commerce website. In the previous version of work, we only worked with the registered users of ecommerce websites. We tracked their mouse movement and clicked information for modeling their interest in database.  Based on modeled data, we provided them recommendation so that they can easily find their product of interest. In this version of work, we suggested generalized algorithms which will support any kind of web applications. User interest data will be stored in the browser cookie rather than database which will improve the loading time of application and overall performance. More detailed brief of data collection process and content reordering is provided. We have compared some exiting approaches defining their problems and solving those issues with our system. This paper mainly focuses on how web application content can be organized in a manner that responds to user interest. For achieving this goal, this paper will focus on providing algorithms, observe user engagement, find user interest, store data in browser cookie, re-organize the web application contents based on interest data.

3.       Overview of Previous Work

      Previous approach that was taken earlier was to design a recommendation model based on the interests of users of ecommerce web applications. The overall process was to learn about the interests of users through mouse-tracking, analyze the interest data to create user model and recommend user according to the best match of their interest using. To complete the process, we proposed two algorithms: Modeling Algorithm (MA) & Recommendation Algorithm (RA). Modeling algorithm is used to collect user interests data through some specific button click (like search button, compare button & detail button) and mouse movement over various content. These information are stored in database. Recommendation algorithm uses these data from database and try to find out any matched product from the intersection of clicked data and mouse movement data and provide recommendation based on obtained product/products category/categories. On finding no matched product at first, it checks for maximum clicked data as it reflect more user interest specifically and recommends product according to the category of that/those product/products. If this happens that user have not yet clicked anything, then it checks for maximum mouse movement data and recommends product/products according to this. If both mouse clicked and movement data are null, then it recommends nothing but the default items.

4.       Proposed Algorithm

4.1.  User Interest Capturing Algorithm (UICA)

      UICA or User Interest Capturing Algorithm is used to observe user behavior in a web application to capture user interest and store interest data in browser cookie. A web application has several content section. We mark each of them with id. At first, we find out the type of activity in which user is involved. In our case, there were two possible scenarios. One is button or hyperlink click and the other one is mouse movement over content. Each of the activity is stored in different way (different array for both click information and moue move information) in browser cookie. Whenever this section is clicked or mouse moved over the content for threshold time, we get that id. Figure 1 shows the code snippet how we identified each of the content.

• Code snippet of identification of a content

      After getting the content id depending on what type of activity user is involved in, we find out whether that item existed in the browser cookie or not. Upon earlier existence we increased the counter of that product. If the item is not already stored in the then we store it in the browser cookie on relevant place (for mouse movement we store the information in cookied_mousemove_data array and for click activity we store the information in cookied_click_data array).  Details discussion about the whole process is presented in the implementation section of this paper.

     Figure 2 shows the visualization of how mouse data is stored

• Visualization of how mouse data is stored.

      Figure 3 shows User Interest Capturing Algorithm.

• User Interest Capturing Algorithm

4.2.  Content Reordering Algorithm (CRA)

CRA or Content Reordering Algorithm as the name suggest is used for content reorganization based on browser cookie data which was collected through UICA. The process of reordering is discussed below:

      First we get all the mouse moved and content clicked data from browser cookie. We find out the max clicked and max mouse moved item/items id/ids. Assuming that both of the max clicked and mouse moved items are not empty, the system will find out intersection of both information and try to find out related items. Finding upon any related items the content will be reorganized or else default content will be served to the user. If any of the mouse moved and clicked data is empty then the system will find out related items based on the kind of data that was available. When any related item/items is found, then the system will automatically reorganize its content accordingly.  If there is no information stored in the cookie both for mouse moved and clicked data than the default content will be displayed. More brief discussion is covered in the Implementation section.

Figure 4 shows Content Reordering Algorithm.

• Content Reordering Algorithm

5.       Proposed Approach for Web Applications

      Web applications engage users. Web interfaces that has confusing navigation and huge content bores the user. People often avoid these applications and go elsewhere to perform their tasks. The most challenging task for any web application is how to design so that it can satisfy all of its users. Successful web applications serves appropriate content which stimulate and maintain the users interest, motivation, and attention through relevant and exquisite information related to individual. In short, they ease users enjoyment of their tasks. User engagement (UE) and user experience can help user in judging product quality during interaction [9]. Our approach will solve this problem as discussed in previous section.

      For any kind of user-centered application, it is mandatory to know about what each user interested in. User interest is very difficult to find out as it varies with age and culture. Click data can identify user interest [10]. Interest model can be built upon user reading and organizing of documents [11]. Interest of users can be known explicitly or implicitly. Explicit learning includes feedback from users or the rating system. As there is no benefit of giving feedback or rating has no influence on their user experience they may not give any rating [12]. In implicit learning procedure users remain completely unaware of the fact that they are giving information about their liked contents. Mouse scroll movement and time spent any content can be hint of user’s interests [13]-[14].

      Eye tracking is the best possible way to learn about user interest for this reason human eye has been studied in HCI [15]. The advertising industry, entertainment industry, packaging industry and web design [16] can be significantly benefited from studying the visual behavior of the user. The automotive, medical and defense industries have also been using eye tracking technology to make us safer. Understanding of how eye moves has become an extremely important consideration in research and design. There are some problems in eye-tracking. They are:

• Eye-tracking equipment’s are costly.
• For getting correct data these devices need proper alignment with user’s eyes. As user’s eye position cannot be fixed to a place as body moves so the task becomes more difficult.
• If one or both eyes do not move smoothly, accurately and quickly across a line or from one object to another it creates problems. This problem includes the inability to fixate (lock one’s eyes) onto a single target (like a word on a printed page).
• If a person’s word to word eye movement is not tracked precisely then it creates major problems like reading, writing or copying.

The remedy of this problem lies in mouse tracking. As users have to use mouse to interact with any web application, it is the best and most cost effective way to find out user interest from their mouse movement. Earlier studies shows that how mouse movement is related to eye movement [17]-[20]. We would like to mention here that this paper is beyond the debate of the relationship between eye movement and mouse movement. The paper [21] represents an estimation of visual attention on Search Engine Result Pages (SERP) by log-based mouse movement model. This includes snippets, aggregated results and advertisements. The mouse movement and click data can reflect user’s behavior while interaction [8], [22], [23]. Previous research shows that user interest can be traced by clicking, reading, hesitating, mouse hovering and scrolling [24]-[27]. Biometric identification can be done by mouse movement [28]. It is essential for any web application to learn whether existing user interface provides support to different kinds of users [29]. The effectiveness, efficiency and usability of the website is improved by the involvement of the users in design and evaluation [30]. In the field of web marketing mouse tracking plays a vital role as it improves user experience in web applications by effective mouse movement and user hesitation levels [31].

Any software usually uses database to store information. Large scale application contains huge number of information offering to its users. In such situation it would not be feasible to use database to store user interest information as it can lead to crash or the application being slow. Other possible way is browser cookies.

User preferences, shopping carts information and anonymous data of third party applications like Google Analytic etc. are retained in cookies. User authentication is made dynamic and validated with the help of browser cookies [32].  Cookies also make user browsing experience better. The paper [33] defines how structured data should be stored in browser.

      Continuous growth of web application has led to overload problem both for information and its usage. User often finds it difficult to find their necessary information when it is needed the most [34]-[35]. Recommendation system identifies the problem and comes up with a solution by finding user interest data from large number of contents [35]-[36].  Recommendation systems use details of user’s views, opinions, liking, disliking, habits etc. to find out the best match information that is nearly or in some case exactly the same as their interests..

6.       Implementation

Implementation steps are followed as below:

6.1.  Demo Website

First and the most obvious necessary thing is a website. We developed a e-commerce web application for our testing with an attractive outlook.

6.2.  Data Collection

The data were collected from user implicitly. We asked users to visit our system. System will then observed user behavior while interacting with our system and reorganize its contents according to user specific behavior. For knowing user interest we used our User Interest Capturing Algorithm (UICA). The whole process of data collection are described below:.

• Click Data Collection: In any kind of interaction with the web application user click is a very important feature. As user is intended to click the things that attracts them most or the things that fulfill their needs, user clicked data were taken into consideration in our content reordering. Our system will check if user have clicked on button or hyperlink or not, if so then it will search for that content in the browser cookie where all clicked data are stored. If any match is found then system will increase the number of click of that item otherwise it will add the content in the clicked data as a new entry. The below figure shows how button click data will be stored in the cookie. Figure 5 how button click data will be stored in the cookie.
• Mouse Data Collection: What if user only browses a system without clicking any content? User can only read the summery in this situation. If user mouse enters any specific content over and over again, this indicates that user is interested in that specific content. This also should be taken into consideration while collecting user interest information. Mouse movement data are collected on user’s each movement. We defined a threshold time in our case it is 1 sec. If at any time user mouse remains still over a content for the threshold time then we will take that action in our consideration and check if the content is already in the mouse movement data in the cookie. If this happens then we will increase the number of mouse movement data of that item otherwise it will add the content in the mouse movement data as a new entry. Figure 6 shows how user mouse movement data would be stored in the browser cookie.

6.3.  Content Reordering

For content reordering our developed system will use Content Reordering Algorithm (CRA). As user mouse may remain still at some content even if user has no interest in that content. This will not provide us with an actual solution. That is why we will mainly focus on user clicked data.

• Process of collecting user clicked data.
• Process of collecting user mouse movement data.

The total process of content reordering is given below in brief:

• First of all the system will get all the clicked and mouse movement data from cookie.
• Finding out the max clicked and max mouse moved items from the cookie is the next step.
• The next step of our procedure is to check whether both of the max clicked and max mouse moved items are empty or not. When this happens the system will then find the intersection of max clicked and max mouse moved items. On finding any item or items then it organizes its contents in manner that responses to the related items to the item or items from the intersection. If it fails to find any item, the system will not change or reorder its contents at all.
• On the failing of above situation system will check out for max clicked items. If it finds one or more item/items then content of the web application will adapt to the situation by reordering its contents.
• Whenever step 3 and step 4 fails then the system will prioritize on the max moused moved items. In the state where the system finds one or more items, then it will reorder its contents related to those item/items. Otherwise, there will be no reordering.
• Lastly if a situation arises that user visits the application for the first time or above step 3, 4, 5 don’t provide with the related item/items required then the content will remain the same as it was earlier.

7.       Result Analysis

For evaluating our algorithms and data collection process our demo website was put into a web server. That was there for about a month. We asked the visitors to browse our application for 5-10 minutes. By this time using our UICA, we collected user interest data and system adapted to its contents using CRA. We also asked users explicitly about their liking’s (contents) in our system for better understanding of the efficiency of our proposed algorithm UICA and CRA.

Over the period of a month data was collected from 120 users of different age and profession. Figure 7 shows rating from anonymous users. Over all findings are quite satisfactory. Rating from user averaged 4.43 out of 5.00.

• User Ratings (anonymous)

The system incredibly got accuracy of 91.67% (110 out of 120 users) of serving user according to their interests. Figure 8 shows the accuracy of our evaluation.

For better understanding of our approach we will briefly describe content reorganization for a random specific user among the 120 users. Figure 9 shows the content before user browse our system. The user is a businessman aged around 42 years. He visited our developed e-commerce web application. Figure 10 shows the visualization of his interaction with the system.

While he was interacting with the system, system closely monitor his mouse movement and click on web content. We stored the type id/ids in our case the category of the product both for mouse movement and click in browser. Figure 11 shows how data is stored in user’s browser and findings of user interests.

Reorganization of content is done with the help of CRA. Using the algorithm, the system found out that he has mostly visited the electronic goods or clothing type items (type/category id 7 and 15) which was the intersection of mouse moved data and clicked. The system than reorganized its products which were of electronics and clothing (serve products of electronics category first as it has the most count value which was acquired from the sum of click and mouse moved count value). Figure 12 shows the content after reordering based on his interest data stored in the browser cookie.

Figure 10    Visulaization of user interaction with the system

Figure 11    Data storage and user interest finding process

• Content after user browsing

      At the completion of his browsing we asked him for his experience. He told that he was overwhelmed to see contents being changed almost immediately after his each activity. He gave us rating of 4.75 out of 5.00.

8.       Comparative Analysis

The paper [8] introduced mouse movements as an additional layer of information that interprets user interest. In our previous work [1] we implemented the idea. We gave a new approach and algorithms which gives the accuracy of 87.50 %.

Our proposed system [1] solved the problem of e-commerce website by storing registered user interest in database and providing recommendation based on the interest data but can face problem in large scale application if we approach with same approach. With huge number of users the system loading time may increase. This will be quite frustrating for the users. The approach discussed in this paper can solve all the existing problems using our newly given two algorithms UICA, CRA and storing data in browser cookie. This will not affect the performance of the application and there is no need to distinguish between registered or unregistered user anymore. New approach gained accuracy of 91.67% compared to earlier accuracy which was 87.50%. So the accuracy has increased 4.17%. Figure 13 shows the accuracy of different systems.

There was 1.2% improvement in the rating system. Our previous system loading time was 2.8749 sec where new system loads in 2.6531 sec leading to 22.14% improvement of performance. Figure 14 shows the improvement of performance using our new approach.

Figure 13    Accuracy comparison graph

Figure 14    Loading time comparison graph

9.       Conclusion

Purpose of this research is to make web applications more user oriented for better interaction and better user experience. Our proposed approach is hassle free as our system will observe user’s common actions (mouse movement and click) without bothering them and reorganize its contents according to their needs. This approach can be helpful for both large and small scale web applications. Our system also provided great accuracy in identifying user’s interests and reordering its content according to each user interest. We would like to introduce machine learning to raise the accuracy level in future.

Conflict of Interest

The authors declare no conflict of interest.

1. Md. Tanjim-Al-Akib, Lutfullahil Kabir Ashik, and Krishanu Chowdhury, “User-modeling and recommendation based on mouse-tracking for e-commerce websites,” Computer and Information Technology (ICCIT), 2016 19th International Conference on. IEEE, 2016. https://doi.org/ 10.1109/ICCITECHN.2016.7860252.
2. Polona Selic, Eva Cedilnik Gorup, Savin Gorup, Marija Petek Ster, Janez Rifel and Zalika Klemenc Ketis, “The Effects of a Web Application and Medical Monitoring on the Quality of Medication, Adverse Drug Events and Adherence in the Elderly Living at Home: a Protocol of the Study” Materia socio-medica 28, no. 6 (2016): 432-436. https://doi.org/msm.2016.28.432-436.
3. Song, Hengjie, Ruoxue Liao, Xiangliang Zhang, Chunyan Miao, and Qiang Yang, “A Mouse-Trajectory Based Model for Predicting Query-URL Relevance” in Twenty-Sixth AAAI Conference on Artificial Intelligence. 2012.https://www.aaai.org/ocs/index.php/AAAI/AAAI12/paper/viewPaper/4968
4. Olivier Chapelle, and Ya Zhang, “A dynamic bayesian network click model for web search ranking” in Proceedings of 18th international conference on World Wide Web. ACM, 2009. https://doi.org/10.1145/1526709.1526711.
5. Peter Brusilovsky, “Adaptive navigation support”, The adaptive web (2007): 263-290.
6. Bc Josef Kunhart, “Web content management system: architecture, concepts and application,” PhD dissertation, Master Thesis, Czech Technical University, Prague: Department of Computer Science and Engineering, 2013.
7. Hans-Joachim Bieg, Lewis L. Chuang, Roland W. Fleming, Harald Reiterer, and Heinrich H. Bülthoff, “Eye and pointer coordination in search and selection tasks,” in Proceedings of the 2010 Symposium on Eye-Tracking Research & Applications, ACM, 2010. https://doi.org/10.1145/1743666.1743688.
8. Florian Mueller, and Andrea Lockerd, “Cheese: tracking mouse movement activity on websites, a tool for user modeling,” CHI’01 extended abstracts on Human factors in computing systems. ACM, 2001, 279-280. https://doi.org/10.1145/634067.634233.
9. Jennefer Hart, Alistair Sutcliffe, and Antonella di Angeli, “Evaluating User Engagement Theory,” in Proceedings of CHI Conference on Human Factors in Computing Systems, 5th to 10th May 2012. http://openlab.ncl.ac.uk/uxtheory/files/2011/11/5_Hart.pdf.
10. Vikas Chahar, “An Analytical Study on HCI” International Journal of Computer Science and Management Studies, 2012, 251-254. http://www.ijcsms.com/journals/Volume%2012,%20Issue%2001,%20January%202012_Volume%2012,%20Issue%2001,%20January%202012_Vikas%20Paper.pdf.
11. Rajiv Badi , Soonil Bae , J. Michael Moore , Konstantinos Meintanis , Anna Zacchi , Haowei Hsieh , Frank Shipman , Catherine C. Marshall, “Recognizing user interest and document value from reading and organizing activities in document triage,” in Proceedings of the 11th international conference on Intelligent user interfaces. ACM, 2006. https://doi.org/10.1145/1111449.1111496.
12. Jonathan Grudin, “Groupware and social dynamics: Eight challenges for developers” Communications of the ACM, 37(1), 1994, 92-105. https://doi.org/10.1145/175222.175230.
13. Philip K. Chan, “A non-invasive learning approach to building web user profiles” in Proceedings of WEBKDD ’99 Revised Papers from the International Workshop on Web Usage Analysis and User Profiling 1999. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.35.2866.
14. Mark Claypool, Phong Le, Makoto Wased, and David Brown, “Implicit interest indicators” in Proceedings of the 6th international conference on Intelligent user interfaces, ACM, 2001. https://doi.org/10.1145/359784.359836.
15. R. J. Jacob and Keith S. Karn, The Mind’s Eye Cognitive and Applied Aspects of Eye Movement Research Vol. 12, Issue 01, 2003.
16. Robert J.K. Jacob, Eye tracking in advanced interface design, Virtual environments and advanced interface design, 1995.
17. Atsuo Murata, Masakazu Higashijima, and Takehito Hayami, “Relationship between trajectory of mouse pointer and eye movements characteristics during search activity,” in Proceedings of SICE Annual Conference (SICE), IEEE, 2012.
18. Daniel J. Liebling, and Susan T. Dumais, “Gaze and mouse coordination in everyday work” in Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, 2014. https://doi.org/10.1145/2638728.2641692.
19. Kerry Rodden, and Xin Fu, “Exploring how mouse movements relate to eye movements on web search results pages” in SIGIR Workshop on Web Information Seeking and Interaction, 29-32, 2007. www.rodden.org/kerry/wisi2007.pdf.
20. Kerry Rodden, Xin Fu, Anne Aula, and Ian Spiro, “Eye-mouse coordination patterns on web search results pages,” in CHI’08 extended abstracts on Human factors in computing systems, 2997-3002. ACM, 2008. https://doi.org/10.1145/1358628.1358797.
21. Fernando Diaz, Ryen White, Georg Buscher, and Dan Liebling, “Robust models of mouse movement on dynamic web search results pages,” in Proceedings of the 22nd ACM international conference on Conference on information & knowledge management, ACM, 2013. https://doi.org/10.1145/2505515.2505717.
22. Hideo Joho, Robert Villa, and J. Jose, “Interaction pool: Towards a user-centered test collection,” in SIGIR’07 Web Information-Seeking and Interaction Workshop, 2007. https://pdfs.semanticscholar.org/44ed/e9d34dda
    8676f118ee76e34d47e66cd2bd01.pdf.
23. Jeff Huang, Ryen W. White, Georg Buscher, and Kuansan Wang, “Improving searcher models using mouse cursor activity,” in Proceedings of SIGIR ’12 Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval, 2012. https://doi.org/ 10.1145/2348283.2348313
24. Luis A. Leiva Torres, and Roberto Vivo Hernando, “A gesture inference methodology for user evaluation based on mouse activity tracking,” in IHCI 2008, Proceedings of the IADIS International Conference on Interfaces and Human Computer Interaction, Amsterdam, 2008. https://smt.speedzinemedia.com/docs/pubs/smt-ihci08.pdf
25. Nazirah Abd Hamid, Suhailan Safei, Siti Dhalila Mohd Satar, Suriayati Chuprat, and Rabiah Ahmad, “Randomized Mouse Movement for Behavioral Biometric Identification,” International Journal of Interactive Digital Media, 1(2), 52-57, 2013. https://ijidm.org/wp-content/uploads/2013-1-2-10-nazirah.pdf
26. Mark D. Smucker, Xiaoyu Sunny Guo, and Andrew Toulis, “Mouse movement during relevance judging: implications for determining user attention,” in Proceedings of the 37th international ACM SIGIR conference on Research & development in information retrieval, ACM, 2014. https://doi.org/10.1145/2600428.2609489
27. Anuradha Gupta, and Vartika Srivastava, “PREDICTING USER INTEREST FROM MOUSE DATA”. http://cerc.iiitd.ac.in/spsymp15/papers/18.pdf.
28. Alfred Kobsa, “Generic user modeling systems” User modeling and user-adapted interaction, 11(1), 49-63, 2001. www.umuai.org/anniversary/2001-UMUAI-kobsa.pdf.
29. Cristina Gena, “Evaluation methodologies and user involvement in user modeling and adaptive systems” User Modeling and User-Adapted Interaction, 74-78, 2003.
30. Cristina Gena, and Stephan Weibelzahl, Usability engineering for the adaptive web, The adaptive web: methods and strategies of web personalization, 2007.
31. Tzafilkou Katerina, Protogeros Nicolaos, and Yakinthos Charalampos, “Mouse tracking for web marketing: enhancing user experience in web application software by measuring self-efficacy and hesitation levels” International Journal of Strategic Innovative Marketing, 1(4), 233-247, 2014. http://dx.doi.org/10.15556/IJSIM.01.04.005.
32. Theodore Jack London Shrader, Garry L. Child, and William H. Gengler, “Dynamic use and validation of HTTP cookies for authentication” U.S. Patent 6,374,359, issued April 16, 2002.
33. Eric A Benson, “Use of browser cookies to store structured data,” U.S. Patent No. 6,714,926. 30 Mar. 2004.
34. Paul Resnick, Neophytos Iacovou, Mitesh Suchak, Peter Bergstrom, and John Riedl, “GroupLens: an open architecture for collaborative filtering of netnews” in Proceedings of the 1994 ACM conference on Computer supported cooperative work, ACM, 1994. https://doi.org/10.1145/192844.192905.
35. Magnus Mortensen, “Design and evaluation of a recommender system” Master’s thesis, Universitetet i Tromsø, 2007.
36. Resnick, Paul, and Hal R. Varian, “Recommender systems” Communications of the ACM, 40(3), 56-58, 1997.

(Click to view)

(Click to view)