Artifact for the paper "How many Web APIs does my JavaScript analysis need to support?"

The username and password of the VM is: ubuntu Location of the scripts In the provided artifact, the scripts are located in folder ~/javascript-apis. Please cd into that folder now, the rest of this guide assumes the commands are launched from there. cd ~/javascript-apis Check the list of websites to use File top-1m.csv contains the websites that will be used by the rest of this artifact. This is a simple CSV file with the format: , . Run the crawler You can run crawler as follows: aa-exec --profile=chrome node scrape_and_parse.js -n 100 -s 0 -o out-100 Where: aa-exec --profile=chrome Makes sure node can use an AppArmor profile needed for Chromium's sandbox scrape_and_parse.js This is the script that performs the crawling -s 0 Skip the first 0 pages, useful for splitting the crawling in several steps, as described in the paper (Section 4.1) -n 100 Gets the first 100 pages -o out-100 Saves the results in folder out-100 Structure of output folder scraper/coverage-stats.csv Information about the status of each particular URL loaded. Format: , , , Example: gogleapis.com,Error: Ret code 404,0.682,"1/22/2026, 3:51:20 PM" facebook.com,OK,2.686,"1/22/2026, 3:51:22 PM" .js All the JavaScript loaded for a particular URL .json All the WebAPI methods and events found, executed or not .range.json Coverage, obtained from Chromium, when loading the JavaScript screenshots/ .png Screenshot of the page loaded for URL Activate pip environment To run any Python scripts, you need to active the pip environment first by running the command: source tf_env/bin/activate The provided pip environment has the correct versions installed for all the packages needed. Compute the errors To compute the types of errors found when crawling (Table 1 in the paper), use the following script: ./error_stats.py out-100 Where: error_stats.py The script that parses the error file located in /scraper/coverage-stats.csv and prints the results out-100 The location of the files saved during crawling (i.e., in the line above) Screenshots Extract the features of the screenshots First Script extract_features.py processes the screen-shots and extracts the features as described in the paper (Section 3.4). The script can be invoked as: ./extract_features.py out-100 features-100 Where: extract_features.py This is the script that extracts the features from the screenshots out-100 This is the directory where the crawler saved its results. This script looks for a /screenshots directory inside. features-100 This is the file where the extracted features are saved, which will be used for clustering Cluster the features To cluster the features, you can use the following: ./cluster_features.py cluster.json features-100 Where: cluster_features This is the script that will cluster the screenshots based on the features. You can adjust the epsilon on Line 128 It is provided as clusters=cluster(2.5, 2, data) Use an epsilon of 2.5 Use clusters of at least 2 elements cluster.json The script will create this file with all clusters. This file can be inspected as it simply contains groups of URLs clustered together Running ./cluster_stats_json.py cluster.json prints the size of each cluster, and the total number of duplicates found Running ./preview_clusters.py cluster.json out-100 creates a new folder preview that contains one sample of each cluster, copied from the original data folder out-100 We recommend using program thunar to visualize the results as thumbnails features-100 This is the file that contains the features generated in the previous step Finding the best epsilon To run the experiment described in Section 4.3, you can use: python3 DBSCAN.py features-100 out-100 results Where: DBSCAN.py is the script that runs the experiment You can adjust the different epsilon to try on Line 178: epsOptions = [1 , 1.5 , 2, 2.5, 3 , 3.5] You can adjust the maximum "noise" samples to keep in Line 182: maxNoiseSize = 500 features-100 This is the file that contains the features generated in the previous step out-100 This is the directory where the crawler saved its results. This script looks for a /screenshots directory inside. results This script will create a new directory here with the results * Inside this directory you can find folders with the name _ _ where: is the epsilon used is the minimum number of samples per cluster (we recommend 2) is the size of all the screenshots Inside each directory, you can find one directory containing: cluster_N All the screenshots clustered together noise All the screenshots not clustered, up to the configuration above preview One screenshot per cluster We recommend using program thunar to visualize the results as thumbnails WebAPI counts To count the WebAPIs per website, you can use: ./processCounts.py cluster.json counts-100 out-100 Where: processCounts.py is the script that counts the APIs for each website cluster.json is the clustering of visually similar screenshots, obtained from previous steps counts-100 is a file that processCounts.py generates, associating each website with the WebAPIs found/executed out-100 is the folder that has the data crawled in the first step WebAPIs per website (Table 2) To get the information for Table 2, you can run the script: ./processCountsStats.py counts-100 executed Where: processCountsStats.py is the script that counts how many WebAPIs per site counts-100 is the file generated in a previous step, with the WebAPIs per website executed the last argument specifies whether the scripts counts the WebAPIs executed or found while loading the page Combinations of WebAPIs (Tables 3 and 4) To compute how the combinations of WebAPIs supported impacts the number of websites that use that combination, you can use: ./combosTable.py counts-100 executed 5 20 Where: combosTable.py is the script that computes the combinations counts-100 is the file generated in a previous step, with the WebAPIs per website executed specifies whether the scripts counts the WebAPIs executed or found while loading the page 5 20 how many combinations to consider (5) out of the top 20 most popular websites The output looks like: 0 1 0.00 26.168 26.168 set() set() [] 1 20 0.00 38.629 12.461 {'UI Events'} set() ['UI Events'] 2 190 3.00 45.483 6.854 {'Touch Events'} set() ['UI Events', 'Touch Events'] 3 1140 31.00 51.090 5.607 {'History API'} set() ['UI Events', 'Touch Events', 'History API'] Each row contains: 1 Number of WebAPIs considered 20 Number of total combinations computed 0.00 Time to compute those combinations 38.629 Percentage of websites that only use these many WebAPIs 12.461 Difference in percentage to the previous row {'UI Events} WebAPIs added in this row set() WebAPIs removed in this row ['UI Events'] WebAPIs considered in this row