Changeset 371

Show
Ignore:
Timestamp:
08/12/08 21:25:20 (10 years ago)
Author:
cseifert
Message:

added a bunch of documentation

Location:
capture-hpc/trunk
Files:
6 modified

Legend:

Unmodified
Added
Removed
  • capture-hpc/trunk/ReleaseNotes-HPC.txt

    r369 r371  
    77------- 
    88+ added preprocessor plugin architecture. Preprocessor plugins allow to handle the input urls before they are passed onto capture. For instance, this could be used to create a crawler or filtering plugin. 
    9 + added processor ids for state changes (this value is only set if the client plug-in supports this). This allows the client plug-in to determine what URL the attack originates even if multiple URLs are visited at once. The internetexplorerbulk plug-in takes advantage of this functionality. 
     9+ added processor ids for state changes (this value is only set if the client plug-in supports this). This allows the client plug-in to determine what URL the attack originates even if multiple URLs are visited at once.  
     10+ Added internetexplorerbulk plug-in that takes advantage of the processor id functionality. Allows to run multiple URLs without the need to revisit the URLs. A mapping of the state changes to the process id will determine which URL was malicious. 
    1011+ modified client plug-in to communicate the algorithm it is able to support (Divide-and-conquer, bulk, sequential) 
    1112+ upgraded vmware server to 1.0.6, java 6 update 7, NSIS 2.38, boost 1.35.0, visual studio 2008 (requires new VC++ Redist Libraries!) 
  • capture-hpc/trunk/capture-client/ReadMe-HPC.txt

    r365 r371  
    53534.2. Client Configuration 
    5454------------------------- 
    55 In addition to the exclusion list, one can configure the client identifiers of the client honeypot in the applications.conf file. The client identifiers are pointers to client applications that can be used to interact with a server, such as Internet Explorer, Firefox, Opera, but also other applications that can retrieve content from a network, such as Microsoft Word, Adobe Acrobat Reader. Simply add the client identfier and fully qualifying path name to the client.conf file and specify the client identifier with the uri that should be interacted with in the uri list that is piped to the server, e.g. http://www.google.com::firefox.  Note that the client identifier needs to be applied to all the URLs in the input file. Otherwise unpredictable behavior might result. 
     55In addition to the exclusion list, one can configure the client identifiers of the client honeypot in the applications.conf file. The client identifiers are pointers to client applications that can be used to interact with a server, such as Internet Explorer, Firefox, Opera, but also other applications that can retrieve content from a network, such as Microsoft Word, Adobe Acrobat Reader. Simply add the client identfier and fully qualifying path name to the client.conf file and specify the client identifier in the capture-server config file.   
    5656 
    5757Since some applications (like Adobe Acrobat) are unable to retrieve content files directly, Capture might retrieve the files, save them to a temporary folder, and then open the file with the application. To enable such behavior, edit the applications.conf file and specify "yes" at the end of the line. 
    5858 
    59 The applications.conf file settings can be overwritten by a application specific OLE plugin. We provide such a plugin for Internet Explorer. The advantage of the OLE plug-in is its ability to relay more information between the capture client and the client application itself. As such, with the plug-in, it is possible to obtain information on when the page has been successfully loaded. This enables us to dynamically adjust the visitation delay to be page-load-time + a fixed visitation delay (this stands in contrast to using client applications directly where the fixed visitation delay has to accommodate the page-load-time). In addition, the plug-in allows to relay HTTP response code back to the capture-client and server. This will allow one to assess which pages loaded successful, which failed and then which of the successful pages solicited malicious behavior. 
     59When using the client identifiers from the applications.conf files, the group size in the Capture Server configuration needs to be set to 1. An exception is firefox, which supports visiting multiple urls at the same time. 
     60 
     61The applications.conf file settings can be overwritten by specific plugins. We provide three such a plugins. Some of these plugins are a necessity to make capture operate with the corresponding client; others are pure performance enhancers. 
     62 
     63iexplore: This client plugin uses the Internet Explorer component. Its has the ability to relay more information between the capture client and the client application itself. As such, with this plug-in, it is possible to obtain information on when the page has been successfully loaded. This enables us to dynamically adjust the visitation delay to be page-load-time + a fixed visitation delay (this stands in contrast to using client applications directly where the fixed visitation delay has to accommodate the page-load-time). In addition, the plug-in allows to relay HTTP response code back to the capture-client and server. This will allow one to assess which pages loaded successful, which failed and then which of the successful pages solicited malicious behavior. 
     64In addition, the iexplore plug-in allows one to visit multiple URLs at the same time. A divide-and-conquer mechanism is applied in which urls are visited at the same time and when a unauthorized state change is detected, the urls are split into two and revisited until the responsible url is identified. Note, that there is a danger of evasion when directly connecting to the internet, since malicious urls might only trigger on initial contact, but not on subsequent ones. This risk can be reduced by using a proxy that caches all content. 
     65 
     66iexplorebulk: This is a client plugin that opens Internet Explorer in its own process. It has the same advantages as iexplore in collecting additional information about interacting with a URL. It is also able to visit multiple URLs are the same time. However, because each Internet Explorer instance is started in its own process, state changes can be mapped to the responsible URL without re-visitation. On the flipside, iexplorebulk uses more resources, because individual instances are created. Also, the malware and network collection does not map to specific URLs, but rather to a group of URLs. Manual filtering of the data would be required to map malware and network traffic to a specific URL.  
     67 
     68safari: This is a simple plugin for Apple's safari browser. Because safari doesnt support visitation of a URL by specifying it on the command line (e.g. safari.exe http://www.foo.com), the applications.conf mechanism could not be used, because it conveys information about which URL to visit via the command line. This necessitated the implementation of a custom plug-in, which sets the homepage (located on a file on disk) prior to opening safari. Because of this mechanism, safari can only visit one URL at a time. 
    6069 
    61705. Post installation 
  • capture-hpc/trunk/capture-server/Readme.txt

    r367 r371  
    3232  - same_vm_revert_delay: the vix library and vmware server have a difficult time reverting vms at the same time. the code already prevents the same VM from reverting at the same time. the delay specified by this variable is automatically applied when reverting the same vm. 
    3333  - different_vm_revert_delay: the vix library and vmware server have a difficult time reverting vms at the same time. the delay specified by this variable is automatically applied when reverting a different vm. This delay is larger because theoretically it would be possible to delay two VMs at the same time. 
    34 * The global option group size determines how many instances of the client application are opened at the same time. A value of 1 will cause only 1 instance to be opened (just like Capture-HPC v 2.01 and prior). Note only certain client plug-ins support this feature: 
     34* The global option group size determines how many instances of the client application are opened at the same time. A value of 1 will cause only 1 instance to be opened (just like Capture-HPC v 2.01 and prior). Note only certain client plug-ins support visiting group of sizes larger than one: 
    3535        - internetexplorer (applies divide-and-conquer algorithm): full support (max group size of 80) 
    3636        - internetexplorerbulk (applies bulk algorithm): full support (max group size of 54) 
    3737        - Firefox (applies divide-and-conquer algorithm): full support; however, firefox needs to be configured to open a blank page and not restore from previous sessions. In addition, because firefox does not have a callback that notifies the server when a page has successfully been retrieved, the client-default-visit-time needs to be increased to accommodate loading X firefox instances and retrieving the web pages. Some testing might be required to determine the appropriate value. 
    38         - Other: not supported at this point 
     38        - Other: only group sizes of 1 are supported at this point 
    3939* Add the local exclusion lists that would be pushed to the clients if that option is enabled  
    4040* Add vmware servers 
     
    8585Example: java -Djava.net.preferIPv4Stack=true -jar CaptureServer.jar -s <IP listening address>:<IP listening port> -f input_uris.txt. 
    8686 
    87 One can specify a specific client application to have Capture client to visit a server with (The default is set via the client-default global property in the config.xml. By default it is set to Internet Explorer). This occurs by appending a client idenifier separated by two colons after the URI. Also one can overwrite the default visitation time, for example, http://www.google.com::firefox::45. The client identifier needs to be specified in the applications.conf on the client side and point to the executable of the client application. (see the Capture Client readme.txt for more information) 
     87One can specify a specific client application to have Capture client to visit a server with (The default is set via the client-default global property in the config.xml. By default it is set to Internet Explorer Bulk). This occurs by appending a client identifier separated by two colons after the URI. Also one can overwrite the default visitation time, for example, http://www.google.com::firefox::45. The client identifier needs to be specified in the applications.conf on the client side and point to the executable of the client application. When group size is configured to be larger than 1, it is not recommended to overwrite the visitiation time and client. (see the Capture Client readme.txt for more information) 
    8888 
    89894.3 Report Description 
  • capture-hpc/trunk/capture-server/build.xml

    r340 r371  
    6262                <copy file="./COPYING" todir="${release}"/> 
    6363                <copy file="./Readme.txt" todir="${release}"/> 
     64                <copy file="./preprocessor_README.txt" todir="${release}"/> 
    6465                <copy file="./input_urls_example.txt" todir="${release}"/> 
    6566                <copy file="./config.xsd" todir="${release}"/> 
  • capture-hpc/trunk/capture-server/config.xml

    r370 r371  
    33        <!-- version 2.5 --> 
    44        <global collect-modified-files="false"  
    5                         client-default="iexplorerbulk" 
     5                        client-default="iexplorebulk" 
    66                        client-default-visit-time="20" 
    77                        capture-network-packets-malicious="false" 
  • capture-hpc/trunk/capture-server/preprocessor_README.txt

    r321 r371  
    111. Capture Preprocessor Plug-in Architecture README 
    22--------------------------------------------------- 
    3 The Capture Preprocessor Plug-in Architecture is a feature that was added with version 2.2 of Capture-HPC. It allows one to change the behavior of Capture-HPC in its way to process the input urls. Prior to the existence preprocessors, Capture-HPC simply forwarded all input URLs to the client honeypot for inspection. The preprocessors allow one to adjust this flow. Instead of forwarding the input URLs to Capture-HPC directly, they are first forwarded to the preprocessor. The preprocessor can process the URLs and then forward input URLs to be inspected by the Capture system to its liking. Common examples of preprocessors are crawlers and filters. Currently, only one preprocessor at a time is supported by Capture-HPC. 
     3The Capture Preprocessor Plug-in Architecture is a feature that was added with version 2.5 of Capture-HPC. It allows one to change the behavior of Capture-HPC in its way to process the input urls. Prior to the existence preprocessors, Capture-HPC simply forwarded all input URLs to the client honeypot for inspection. The preprocessors allow one to adjust this flow. Instead of forwarding the input URLs to Capture-HPC directly, they are first forwarded to the preprocessor. The preprocessor can process the URLs and then forward input URLs to be inspected by the Capture system to its liking. Common examples of preprocessors are crawlers and filters. Currently, only one preprocessor at a time is supported by Capture-HPC (this might change in the future if there is a need). 
    44 
    55Preprocessors are configured and instantiated based on the preprocessor directive in the config.xml file. Capture will use the dynamic class loader to instantiate the preprocessor using the classname as the fully qualifying class name. The preprocessor plug-in’s configuration is provided via CDATA of the preprocessor tag.