diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..4dfd2b2 --- /dev/null +++ b/.gitignore @@ -0,0 +1,12 @@ +target/ +sootOutput/ +out2/ +.idea/* +config/zbconfig.json +config/bzxconfig.json5 +tools/server/ +.gradle/* +build/* +.DS_Store +flowdroidAndSoot.src/* +out/ \ No newline at end of file diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..49c406d --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,60 @@ +# How to Contribute + +## Your First Pull Request + +We use GitHub for our codebase. You can start by +reading [How To Pull Request](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/about-pull-requests) +. + +## Without Semantic Versioning + +We keep the stable code in branch `master`. Development base on branch `develop`. + +## Bugs + +### 1. How to Find Known Issues + +We are using [Github Issues](https://github.com/bytedance/appshark/issues) for our public bugs. We keep a close eye on +this and try to make it clear when we have an internal fix in progress. Before filing a new task, try to make sure your +problem doesn’t already exist. + +### 2. Security Bugs + +Please do not report the safe disclosure of bugs to public issues. Contact us +by [Support Email](mailto:appshark@bytedance.com) + +## How to Get in Touch + +- [Email](mailto:baizhenxuan@bytedance.com) + +## Submit a Pull Request + +Before you submit your Pull Request (PR) consider the following guidelines: + +1. Search [GitHub](https://github.com/bytedance/appshark/pulls) for an open or closed PR that relates to your + submission. You don't want to duplicate existing efforts. +2. Be sure that an issue describes the problem you're fixing, or documents the design for the feature you'd like to add. + Discussing the design upfront helps to ensure that we're ready to accept your work. +3. [Fork](https://docs.github.com/en/github/getting-started-with-github/fork-a-repo) the bytedance/appshark repo. +4. In your forked repository, make your changes in a new git branch: + ``` + git checkout -b bugfix/security_bug develop + ``` +5. Create your patch, including appropriate test cases. +6. Follow our [Style Guides](#code-style-guides). +7. Push your branch to GitHub: + ``` + git push origin bugfix/security_bug + ``` +8. In GitHub, send a pull request to `appshark:master` + +Note: you must use one of `optimize/feature/bugfix/doc/ci/test/refactor` following a slash(`/`) as the branch prefix. + +## Contribution Prerequisites + +- You are familiar with [Github](https://github.com) +- Maybe you need familiar with [Actions](https://github.com/features/actions)(our default workflow tool). + +## Code Style Guides + +See [Coding conventions](https://kotlinlang.org/docs/coding-conventions.html). \ No newline at end of file diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..f433b1a --- /dev/null +++ b/LICENSE @@ -0,0 +1,177 @@ + + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS diff --git a/README.md b/README.md new file mode 100644 index 0000000..9945237 --- /dev/null +++ b/README.md @@ -0,0 +1,138 @@ +# AppShark + +Appshark is a static taint analysis platform to scan vulnerabilities in an Android app. + +## Prerequisites + +Appshark requires a specific version of JDK +-- [JDK 11](https://www.oracle.com/java/technologies/javase/jdk11-archive-downloads.html). After testing, it does not +work on other LTS versions, JDK 8 and JDK 16, due to the dependency compatibility issue. + +## Building/Compiling AppShark + +We assume that you are working in the root directory of the project repo. You can build the whole project with +the [gradle](https://gradle.org/) tool. + +```shell +$ ./gradlew build -x test +``` + +After executing the above command, you will see an artifact file `AppShark-0.1-all.jar` in the directory `build/libs`. + +## Running AppShark + +Like the previous step, we assume that you are still in the root folder of the project. You can run the tool with + + ```shell + $ java -jar build/libs/AppShark-0.1-all.jar config/config.json5 + ``` + +The `config.json5` has the following configuration contents. + +```JSON +{ + "apkPath": "/Users/apks/app1.apk", + "out": "out", + "rules": "unZipSlip.json", + "maxPointerAnalyzeTime": 600 +} +``` + +Each JSON field is explained below. + +- apkPath: the path of the apk file to analyze +- out: the path of the output directory +- rules: the path(s) of the rule file(s), can be more than 1 rules +- maxPointerAnalyzeTime: the timeout duration in seconds set for the analysis started from an entry point +- debugRule: specify the rule name that enables logging for debugging + +If you provide a configuration JSON file which sets the output path as `out` in the project root directory, you will +find the result file `out/results.json` after running the analysis. + +## Interpreting the Results + +Below is an example of the `results.json`. + +```JSON +{ + "AppInfo": { + "AppName": "test", + "PackageName": "net.bytedance.security.app", + "min_sdk": 17, + "target_sdk": 28, + "versionCode": 1000, + "versionName": "1.0.0" + }, + "SecurityInfo": { + "FileRisk": { + "unZipSlip": { + "category": "FileRisk", + "detail": "", + "model": "2", + "name": "unZipSlip", + "possibility": "4", + "vulners": [ + { + "details": { + "position": "", + "Sink": "->$r31", + "entryMethod": "", + "Source": "->$r3", + "url": "/Volumes/dev/zijie/appshark-opensource/out/vuln/1-unZipSlip.html", + "target": [ + "->$r3", + "pf{obj{:35=>java.lang.StringBuilder}(unknown)->@data}", + "->$r11", + "->$r31" + ] + }, + "hash": "ec57a2a3190677ffe78a0c8aaf58ba5aee4d2247", + "possibility": "4" + }, + { + "details": { + "position": "", + "Sink": "->$r34", + "entryMethod": "", + "Source": "->$r3", + "url": "/Volumes/dev/zijie/appshark-opensource/out/vuln/2-unZipSlip.html", + "target": [ + "->$r3", + "pf{obj{:33=>java.lang.StringBuilder}(unknown)->@data}", + "->$r14", + "->$r34" + ] + }, + "hash": "26c6d6ee704c59949cfef78350a1d9aef04c29ad", + "possibility": "4" + } + ], + "wiki": "", + "deobfApk": "/Volumes/dev/zijie/appshark-opensource/app.apk" + } + } + }, + "DeepLinkInfo": { + }, + "HTTP_API": [ + ], + "JsBridgeInfo": [ + ], + "BasicInfo": { + "ComponentsInfo": { + }, + "JSNativeInterface": [ + ] + }, + "UsePermissions": [ + ], + "DefinePermissions": { + }, + "Profile": "/Volumes/dev/zijie/appshark-opensource/out/vuln/3-profiler.json" +} + +``` + +# License + +AppShark is licensed under the [APACHE LICENSE, VERSION 2.0](http://www.apache.org/licenses/LICENSE-2.0) diff --git a/build.gradle.kts b/build.gradle.kts new file mode 100644 index 0000000..a90da43 --- /dev/null +++ b/build.gradle.kts @@ -0,0 +1,83 @@ +import com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar +import org.jetbrains.kotlin.gradle.tasks.KotlinCompile + + +/* + * This file was generated by the Gradle 'init' task. + */ + +plugins { + `maven-publish` + kotlin("jvm") version "1.6.21" + kotlin("plugin.serialization") version "1.6.21" + application + id("com.github.johnrengelman.shadow") version "7.0.0" +} +tasks.withType { + isZip64 = true + mergeServiceFiles() // # <<< Most important line +} + +repositories { + mavenLocal() + maven { + url = uri("https://repo.maven.apache.org/maven2/") + } + mavenCentral() + maven { url = uri("https://maven.pkg.jetbrains.space/public/p/kotlinx-html/maven") } + maven { url = uri("https://repo1.maven.org/maven2") } +} + +dependencies { + implementation("org.apache.httpcomponents:httpmime:4.5.13") + +// implementation("de.fraunhofer.sit.sse.flowdroid:soot-infoflow:2.10.0") + implementation("io.github.nkbai:soot-infoflow-android:2.10.1") + + implementation("org.eclipse.jdt:org.eclipse.jdt.core:3.24.0") + implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.3.3") + implementation(kotlin("stdlib-jdk8")) + implementation("org.jetbrains.kotlin:kotlin-reflect:1.6.21") + // coroutines + implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.2") + val kotlinxHtmlVersion = "0.7.2" + // include for Common module + implementation("org.jetbrains.kotlinx:kotlinx-html:$kotlinxHtmlVersion") + + // test + testImplementation(platform("org.junit:junit-bom:5.7.2")) + testImplementation("org.junit.jupiter:junit-jupiter") +} +tasks.test { + useJUnitPlatform() + testLogging { + events("passed", "skipped", "failed") + } +} +configurations.all { + resolutionStrategy.dependencySubstitution { + substitute(module("org.osgi.service:org.osgi.service.prefs")).with(module("org.osgi:org.osgi.service.prefs:1.1.2")) + } +} + +group = "net.bytedance.security.app" +version = "0.1" +description = "appshark" +java.sourceCompatibility = JavaVersion.VERSION_1_8 + +publishing { + publications.create("maven") { + from(components["java"]) + } +} +val compileKotlin: KotlinCompile by tasks +compileKotlin.kotlinOptions { + jvmTarget = "1.8" +} +val compileTestKotlin: KotlinCompile by tasks +compileTestKotlin.kotlinOptions { + jvmTarget = "1.8" +} +application { + mainClass.set("net.bytedance.security.app.JavaEntry") +} diff --git a/build.sh b/build.sh new file mode 100755 index 0000000..292aa19 --- /dev/null +++ b/build.sh @@ -0,0 +1,4 @@ +#!/bin/sh +export JAVA_HOME=/usr/local/Cellar/openjdk@11/11.0.12 +export PATH=/usr/local/Cellar/openjdk@11/11.0.12/bin:$PATH +./gradlew build -x test \ No newline at end of file diff --git a/config/EngineConfig.json5 b/config/EngineConfig.json5 new file mode 100644 index 0000000..aa6d524 --- /dev/null +++ b/config/EngineConfig.json5 @@ -0,0 +1,1568 @@ +{ + /// matched functions seem like that they did not exist during analyze + IgnoreList: { + "PackageName": [ + "com.meituan.robust", + "kotlin.jvm.internal.Intrinsics", + "com.bytedance.frameworks.apm.trace.MethodCollector" + ], + "MethodName": [], + "MethodSignature": [] + }, + Callback: { + /* + These functions are automatically called when instances of the following classes are created + */ + "param": { + "java.lang.Runnable": [ + "void run()" + ], + "android.os.AsyncTask": [ + "java.lang.Object doInBackground(java.lang.Object[])", + "void onCancelled()", + "void onPostExecute(java.lang.Object)", + "void onProgressUpdate(java.lang.Object[])", + "void onPreExecute()" + ], + "android.content.BroadcastReceiver": [ + "void onReceive(android.content.Context,android.content.Intent)" + ], + "android.os.Handler": [ + "void handleMessage(android.os.Message)" + ], + "android.view.View$OnClickListener": [ + "*" + ], + "android.content.DialogInterface$OnClickListener": [ + "*" + ], + "android.accessibilityservice.AccessibilityButtonController$AccessibilityButtonCallback": [ + "*" + ], + "android.accessibilityservice.AccessibilityService$GestureResultCallback": [ + "*" + ], + "android.accessibilityservice.AccessibilityService$MagnificationController$OnMagnificationChangedListener": [ + "*" + ], + "android.accessibilityservice.AccessibilityService$SoftKeyboardController$OnShowModeChangedListener": [ + "*" + ], + "android.accessibilityservice.FingerprintGestureController$FingerprintGestureCallback": [ + "*" + ], + "android.accounts.AccountManagerCallback": [ + "*" + ], + "android.accounts.OnAccountsUpdateListener": [ + "*" + ], + "android.animation.Animator$AnimatorListener": [ + "*" + ], + "android.animation.Animator$AnimatorPauseListener": [ + "*" + ], + "android.animation.LayoutTransition$TransitionListener": [ + "*" + ], + "android.animation.TimeAnimator$TimeListener": [ + "*" + ], + "android.animation.ValueAnimator$AnimatorUpdateListener": [ + "*" + ], + "android.app.ActionBar$OnMenuVisibilityListener": [ + "*" + ], + "android.app.ActionBar$OnNavigationListener": [ + "*" + ], + "android.app.ActionBar$TabListener": [ + "*" + ], + "android.app.AlarmManager$OnAlarmListener": [ + "*" + ], + "android.app.AppOpsManager$OnOpChangedListener": [ + "*" + ], + "android.app.Application$ActivityLifecycleCallbacks": [ + "*" + ], + "android.app.Application$OnProvideAssistDataListener": [ + "*" + ], + "android.app.DatePickerDialog$OnDateSetListener": [ + "*" + ], + "android.app.FragmentBreadCrumbs$OnBreadCrumbClickListener": [ + "*" + ], + "android.app.FragmentHostCallback": [ + "*" + ], + "android.app.FragmentManager$OnBackStackChangedListener": [ + "*" + ], + "android.app.KeyguardManager$KeyguardDismissCallback": [ + "*" + ], + "android.app.KeyguardManager$OnKeyguardExitResult": [ + "*" + ], + "android.app.LoaderManager$LoaderCallbacks": [ + "*" + ], + "android.app.PendingIntent$OnFinished": [ + "*" + ], + "android.app.SearchManager$OnCancelListener": [ + "*" + ], + "android.app.SearchManager$OnDismissListener": [ + "*" + ], + "android.app.SharedElementCallback": [ + "*" + ], + "android.app.SharedElementCallback$OnSharedElementsReadyListener": [ + "*" + ], + "android.app.TimePickerDialog$OnTimeSetListener": [ + "*" + ], + "android.app.UiAutomation$OnAccessibilityEventListener": [ + "*" + ], + "android.app.WallpaperManager$OnColorsChangedListener": [ + "*" + ], + "android.app.admin.DevicePolicyManager$OnClearApplicationUserDataListener": [ + "*" + ], + "android.app.usage.NetworkStatsManager$UsageCallback": [ + "*" + ], + "android.bluetooth.BluetoothAdapter$LeScanCallback": [ + "*" + ], + "android.bluetooth.BluetoothGattCallback": [ + "*" + ], + "android.bluetooth.BluetoothGattServerCallback": [ + "*" + ], + "android.bluetooth.BluetoothHealthCallback": [ + "*" + ], + "android.bluetooth.BluetoothHidDevice$Callback": [ + "*" + ], + "android.bluetooth.BluetoothProfile$ServiceListener": [ + "*" + ], + "android.bluetooth.le.AdvertiseCallback": [ + "*" + ], + "android.bluetooth.le.AdvertisingSetCallback": [ + "*" + ], + "android.bluetooth.le.ScanCallback": [ + "*" + ], + "android.companion.CompanionDeviceManager$Callback": [ + "*" + ], + "android.content.ClipboardManager$OnPrimaryClipChangedListener": [ + "*" + ], + "android.content.DialogInterface$OnCancelListener": [ + "*" + ], + "android.content.DialogInterface$OnDismissListener": [ + "*" + ], + "android.content.DialogInterface$OnKeyListener": [ + "*" + ], + "android.content.DialogInterface$OnMultiChoiceClickListener": [ + "*" + ], + "android.content.DialogInterface$OnShowListener": [ + "*" + ], + "android.content.IntentSender$OnFinished": [ + "*" + ], + "android.content.Loader$OnLoadCanceledListener": [ + "*" + ], + "android.content.Loader$OnLoadCompleteListener": [ + "*" + ], + "android.content.SharedPreferences$OnSharedPreferenceChangeListener": [ + "*" + ], + "android.content.SyncStatusObserver": [ + "*" + ], + "android.content.pm.LauncherApps$Callback": [ + "*" + ], + "android.content.pm.PackageInstaller$SessionCallback": [ + "*" + ], + "android.database.sqlite.SQLiteTransactionListener": [ + "*" + ], + "android.drm.DrmManagerClient$OnErrorListener": [ + "*" + ], + "android.drm.DrmManagerClient$OnEventListener": [ + "*" + ], + "android.drm.DrmManagerClient$OnInfoListener": [ + "*" + ], + "android.gesture.GestureOverlayView$OnGestureListener": [ + "*" + ], + "android.gesture.GestureOverlayView$OnGesturePerformedListener": [ + "*" + ], + "android.gesture.GestureOverlayView$OnGesturingListener": [ + "*" + ], + "android.graphics.ImageDecoder$OnHeaderDecodedListener": [ + "*" + ], + "android.graphics.ImageDecoder$OnPartialImageListener": [ + "*" + ], + "android.graphics.SurfaceTexture$OnFrameAvailableListener": [ + "*" + ], + "android.graphics.drawable.Animatable2$AnimationCallback": [ + "*" + ], + "android.graphics.drawable.Drawable$Callback": [ + "*" + ], + "android.graphics.drawable.Icon$OnDrawableLoadedListener": [ + "*" + ], + "android.hardware.Camera$AutoFocusCallback": [ + "*" + ], + "android.hardware.Camera$AutoFocusMoveCallback": [ + "*" + ], + "android.hardware.Camera$ErrorCallback": [ + "*" + ], + "android.hardware.Camera$FaceDetectionListener": [ + "*" + ], + "android.hardware.Camera$OnZoomChangeListener": [ + "*" + ], + "android.hardware.Camera$PictureCallback": [ + "*" + ], + "android.hardware.Camera$PreviewCallback": [ + "*" + ], + "android.hardware.Camera$ShutterCallback": [ + "*" + ], + "android.hardware.SensorEventCallback": [ + "*" + ], + "android.hardware.SensorEventListener": [ + "*" + ], + "android.hardware.SensorListener": [ + "*" + ], + "android.hardware.SensorManager$DynamicSensorCallback": [ + "*" + ], + "android.hardware.TriggerEventListener": [ + "*" + ], + "android.hardware.biometrics.BiometricPrompt$AuthenticationCallback": [ + "*" + ], + "android.hardware.camera2.CameraCaptureSession$CaptureCallback": [ + "*" + ], + "android.hardware.camera2.CameraCaptureSession$StateCallback": [ + "*" + ], + "android.hardware.camera2.CameraDevice$StateCallback": [ + "*" + ], + "android.hardware.camera2.CameraManager$AvailabilityCallback": [ + "*" + ], + "android.hardware.camera2.CameraManager$TorchCallback": [ + "*" + ], + "android.hardware.display.DisplayManager$DisplayListener": [ + "*" + ], + "android.hardware.display.VirtualDisplay$Callback": [ + "*" + ], + "android.hardware.fingerprint.FingerprintManager$AuthenticationCallback": [ + "*" + ], + "android.hardware.input.InputManager$InputDeviceListener": [ + "*" + ], + "android.inputmethodservice.KeyboardView$OnKeyboardActionListener": [ + "*" + ], + "android.location.GnssMeasurementsEvent$Callback": [ + "*" + ], + "android.location.GnssNavigationMessage$Callback": [ + "*" + ], + "android.location.GnssStatus$Callback": [ + "*" + ], + "android.location.GpsStatus$Listener": [ + "*" + ], + "android.location.GpsStatus$NmeaListener": [ + "*" + ], + "android.location.LocationListener": [ + "*" + ], + "android.location.OnNmeaMessageListener": [ + "*" + ], + "android.media.AudioDeviceCallback": [ + "*" + ], + "android.media.AudioManager$AudioPlaybackCallback": [ + "*" + ], + "android.media.AudioManager$AudioRecordingCallback": [ + "*" + ], + "android.media.AudioManager$OnAudioFocusChangeListener": [ + "*" + ], + "android.media.AudioRecord$OnRecordPositionUpdateListener": [ + "*" + ], + "android.media.AudioRecord$OnRoutingChangedListener": [ + "*" + ], + "android.media.AudioRouting$OnRoutingChangedListener": [ + "*" + ], + "android.media.AudioTrack$OnPlaybackPositionUpdateListener": [ + "*" + ], + "android.media.AudioTrack$OnRoutingChangedListener": [ + "*" + ], + "android.media.ImageReader$OnImageAvailableListener": [ + "*" + ], + "android.media.ImageWriter$OnImageReleasedListener": [ + "*" + ], + "android.media.JetPlayer$OnJetEventListener": [ + "*" + ], + "android.media.MediaCas$EventListener": [ + "*" + ], + "android.media.MediaCodec$Callback": [ + "*" + ], + "android.media.MediaCodec$OnFrameRenderedListener": [ + "*" + ], + "android.media.MediaDrm$OnEventListener": [ + "*" + ], + "android.media.MediaDrm$OnExpirationUpdateListener": [ + "*" + ], + "android.media.MediaDrm$OnKeyStatusChangeListener": [ + "*" + ], + "android.media.MediaPlayer$OnBufferingUpdateListener": [ + "*" + ], + "android.media.MediaPlayer$OnCompletionListener": [ + "*" + ], + "android.media.MediaPlayer$OnDrmInfoListener": [ + "*" + ], + "android.media.MediaPlayer$OnDrmPreparedListener": [ + "*" + ], + "android.media.MediaPlayer$OnErrorListener": [ + "*" + ], + "android.media.MediaPlayer$OnInfoListener": [ + "*" + ], + "android.media.MediaPlayer$OnMediaTimeDiscontinuityListener": [ + "*" + ], + "android.media.MediaPlayer$OnPreparedListener": [ + "*" + ], + "android.media.MediaPlayer$OnSeekCompleteListener": [ + "*" + ], + "android.media.MediaPlayer$OnSubtitleDataListener": [ + "*" + ], + "android.media.MediaPlayer$OnTimedMetaDataAvailableListener": [ + "*" + ], + "android.media.MediaPlayer$OnTimedTextListener": [ + "*" + ], + "android.media.MediaPlayer$OnVideoSizeChangedListener": [ + "*" + ], + "android.media.MediaRecorder$OnErrorListener": [ + "*" + ], + "android.media.MediaRecorder$OnInfoListener": [ + "*" + ], + "android.media.MediaRouter$Callback": [ + "*" + ], + "android.media.MediaRouter$SimpleCallback": [ + "*" + ], + "android.media.MediaRouter$VolumeCallback": [ + "*" + ], + "android.media.MediaScannerConnection$MediaScannerConnectionClient": [ + "*" + ], + "android.media.MediaScannerConnection$OnScanCompletedListener": [ + "*" + ], + "android.media.MediaSync$Callback": [ + "*" + ], + "android.media.MediaSync$OnErrorListener": [ + "*" + ], + "android.media.RemoteControlClient$OnGetPlaybackPositionListener": [ + "*" + ], + "android.media.RemoteControlClient$OnMetadataUpdateListener": [ + "*" + ], + "android.media.RemoteControlClient$OnPlaybackPositionUpdateListener": [ + "*" + ], + "android.media.RemoteController$OnClientUpdateListener": [ + "*" + ], + "android.media.SoundPool$OnLoadCompleteListener": [ + "*" + ], + "android.media.audiofx.AudioEffect$OnControlStatusChangeListener": [ + "*" + ], + "android.media.audiofx.AudioEffect$OnEnableStatusChangeListener": [ + "*" + ], + "android.media.audiofx.BassBoost$OnParameterChangeListener": [ + "*" + ], + "android.media.audiofx.EnvironmentalReverb$OnParameterChangeListener": [ + "*" + ], + "android.media.audiofx.Equalizer$OnParameterChangeListener": [ + "*" + ], + "android.media.audiofx.PresetReverb$OnParameterChangeListener": [ + "*" + ], + "android.media.audiofx.Virtualizer$OnParameterChangeListener": [ + "*" + ], + "android.media.audiofx.Visualizer$OnDataCaptureListener": [ + "*" + ], + "android.media.browse.MediaBrowser$ConnectionCallback": [ + "*" + ], + "android.media.browse.MediaBrowser$ItemCallback": [ + "*" + ], + "android.media.browse.MediaBrowser$SubscriptionCallback": [ + "*" + ], + "android.media.effect$EffectUpdateListener": [ + "*" + ], + "android.media.effect.EffectUpdateListener": [ + "*" + ], + "android.media.midi.MidiManager$DeviceCallback": [ + "*" + ], + "android.media.midi.MidiManager$OnDeviceOpenedListener": [ + "*" + ], + "android.media.projection.MediaProjection$Callback": [ + "*" + ], + "android.media.session.MediaController$Callback": [ + "*" + ], + "android.media.session.MediaSession$Callback": [ + "*" + ], + "android.media.session.MediaSessionManager$OnActiveSessionsChangedListener": [ + "*" + ], + "android.media.tv.TvInputManager$TvInputCallback": [ + "*" + ], + "android.media.tv.TvRecordingClient$RecordingCallback": [ + "*" + ], + "android.media.tv.TvView$OnUnhandledInputEventListener": [ + "*" + ], + "android.media.tv.TvView$TimeShiftPositionCallback": [ + "*" + ], + "android.media.tv.TvView$TvInputCallback": [ + "*" + ], + "android.net.ConnectivityManager$NetworkCallback": [ + "*" + ], + "android.net.ConnectivityManager$OnNetworkActiveListener": [ + "*" + ], + "android.net.nsd.NsdManager$DiscoveryListener": [ + "*" + ], + "android.net.nsd.NsdManager$RegistrationListener": [ + "*" + ], + "android.net.nsd.NsdManager$ResolveListener": [ + "*" + ], + "android.net.sip.SipAudioCall$Listener": [ + "*" + ], + "android.net.sip.SipRegistrationListener": [ + "*" + ], + "android.net.sip.SipSession$Listener": [ + "*" + ], + "android.net.wifi.WifiManager$LocalOnlyHotspotCallback": [ + "*" + ], + "android.net.wifi.WifiManager$WpsCallback": [ + "*" + ], + "android.net.wifi.aware.AttachCallback": [ + "*" + ], + "android.net.wifi.aware.DiscoverySessionCallback": [ + "*" + ], + "android.net.wifi.aware.IdentityChangedListener": [ + "*" + ], + "android.net.wifi.p2p.WifiP2pManager$ActionListener": [ + "*" + ], + "android.net.wifi.p2p.WifiP2pManager$ChannelListener": [ + "*" + ], + "android.net.wifi.p2p.WifiP2pManager$ConnectionInfoListener": [ + "*" + ], + "android.net.wifi.p2p.WifiP2pManager$DnsSdServiceResponseListener": [ + "*" + ], + "android.net.wifi.p2p.WifiP2pManager$DnsSdTxtRecordListener": [ + "*" + ], + "android.net.wifi.p2p.WifiP2pManager$GroupInfoListener": [ + "*" + ], + "android.net.wifi.p2p.WifiP2pManager$PeerListListener": [ + "*" + ], + "android.net.wifi.p2p.WifiP2pManager$ServiceResponseListener": [ + "*" + ], + "android.net.wifi.p2p.WifiP2pManager$UpnpServiceResponseListener": [ + "*" + ], + "android.net.wifi.rtt.RangingResultCallback": [ + "*" + ], + "android.nfc.NfcAdapter$CreateBeamUrisCallback": [ + "*" + ], + "android.nfc.NfcAdapter$CreateNdefMessageCallback": [ + "*" + ], + "android.nfc.NfcAdapter$OnNdefPushCompleteCallback": [ + "*" + ], + "android.nfc.NfcAdapter$OnTagRemovedListener": [ + "*" + ], + "android.nfc.NfcAdapter$ReaderCallback": [ + "*" + ], + "android.os.CancellationSignal$OnCancelListener": [ + "*" + ], + "android.os.Handler$Callback": [ + "*" + ], + "android.os.IBinder$DeathRecipient": [ + "*" + ], + "android.os.MessageQueue$IdleHandler": [ + "*" + ], + "android.os.MessageQueue$OnFileDescriptorEventListener": [ + "*" + ], + "android.os.ParcelFileDescriptor$OnCloseListener": [ + "*" + ], + "android.os.ProxyFileDescriptorCallback": [ + "*" + ], + "android.os.RecoverySystem$ProgressListener": [ + "*" + ], + "android.os.StrictMode$OnThreadViolationListener": [ + "*" + ], + "android.os.StrictMode$OnVmViolationListener": [ + "*" + ], + "android.os.storage.OnObbStateChangeListener": [ + "*" + ], + "android.preference.Preference$OnPreferenceChangeListener": [ + "*" + ], + "android.preference.Preference$OnPreferenceClickListener": [ + "*" + ], + "android.preference.PreferenceFragment$OnPreferenceStartFragmentCallback": [ + "*" + ], + "android.preference.PreferenceManager$OnActivityDestroyListener": [ + "*" + ], + "android.preference.PreferenceManager$OnActivityResultListener": [ + "*" + ], + "android.preference.PreferenceManager$OnActivityStopListener": [ + "*" + ], + "android.print.PrintDocumentAdapter$LayoutResultCallback": [ + "*" + ], + "android.print.PrintDocumentAdapter$WriteResultCallback": [ + "*" + ], + "android.printservice.CustomPrinterIconCallback": [ + "*" + ], + "android.provider.FontsContract$FontRequestCallback": [ + "*" + ], + "android.renderscript.Allocation$OnBufferAvailableListener": [ + "*" + ], + "android.sax.ElementListener": [ + "*" + ], + "android.sax.EndElementListener": [ + "*" + ], + "android.sax.EndTextElementListener": [ + "*" + ], + "android.sax.StartElementListener": [ + "*" + ], + "android.sax.TextElementListener": [ + "*" + ], + "android.se.omapi.SEService$OnConnectedListener": [ + "*" + ], + "android.security.ConfirmationCallback": [ + "*" + ], + "android.security.KeyChainAliasCallback": [ + "*" + ], + "android.service.autofill.FillCallback": [ + "*" + ], + "android.service.autofill.SaveCallback": [ + "*" + ], + "android.service.carrier.CarrierMessagingService$ResultCallback": [ + "*" + ], + "android.service.voice.AlwaysOnHotwordDetector$Callback": [ + "*" + ], + "android.speech.RecognitionListener": [ + "*" + ], + "android.speech.RecognitionService$Callback": [ + "*" + ], + "android.speech.tts.SynthesisCallback": [ + "*" + ], + "android.speech.tts.TextToSpeech$OnInitListener": [ + "*" + ], + "android.speech.tts.TextToSpeech$OnUtteranceCompletedListener": [ + "*" + ], + "android.speech.tts.UtteranceProgressListener": [ + "*" + ], + "android.support.v4.app.FragmentManager$OnBackStackChangedListener": [ + "*" + ], + "android.support.v4.content.Loader$OnLoadCompleteListener": [ + "*" + ], + "android.support.v4.view.PagerTitleStrip$PageListener": [ + "*" + ], + "android.support.v4.view.ViewPager$OnAdapterChangeListener": [ + "*" + ], + "android.support.v4.view.ViewPager$OnPageChangeListener": [ + "*" + ], + "android.support.v4.view.ViewPager$SimpleOnPageChangeListener": [ + "*" + ], + "androidx.fragment.app.FragmentManager$OnBackStackChangedListener": [ + "*" + ], + "androidx.loader.content.Loader$OnLoadCompleteListener": [ + "*" + ], + "androidx.viewpager.widget.PagerTitleStrip$PageListener": [ + "*" + ], + "androidx.viewpager.widget.ViewPager$OnAdapterChangeListener": [ + "*" + ], + "androidx.viewpager.widget.ViewPager$OnPageChangeListener": [ + "*" + ], + "androidx.viewpager.widget.ViewPager$SimpleOnPageChangeListener": [ + "*" + ], + "android.telecom.Call$Callback": [ + "*" + ], + "android.telecom.InCallService$VideoCall$Callback": [ + "*" + ], + "android.telecom.RemoteConference$Callback": [ + "*" + ], + "android.telecom.RemoteConnection$Callback": [ + "*" + ], + "android.telecom.RemoteConnection$VideoProvider$Callback": [ + "*" + ], + "android.telephony.PhoneStateListener": [ + "*" + ], + "android.telephony.SubscriptionManager$OnSubscriptionsChangedListener": [ + "*" + ], + "android.telephony.TelephonyManager$UssdResponseCallback": [ + "*" + ], + "android.telephony.TelephonyScanManager$NetworkScanCallback": [ + "*" + ], + "android.telephony.mbms.DownloadProgressListener": [ + "*" + ], + "android.telephony.mbms.DownloadStatusListener": [ + "*" + ], + "android.telephony.mbms.MbmsDownloadSessionCallback": [ + "*" + ], + "android.telephony.mbms.MbmsStreamingSessionCallback": [ + "*" + ], + "android.telephony.mbms.StreamingServiceCallback": [ + "*" + ], + "android.text.TextUtils$EllipsizeCallback": [ + "*" + ], + "android.text.TextWatcher": [ + "*" + ], + "android.text.method.BaseKeyListener": [ + "*" + ], + "android.text.method.DateKeyListener": [ + "*" + ], + "android.text.method.DateTimeKeyListener": [ + "*" + ], + "android.text.method.DialerKeyListener": [ + "*" + ], + "android.text.method.DigitsKeyListener": [ + "*" + ], + "android.text.method.KeyListener": [ + "*" + ], + "android.text.method.MetaKeyKeyListener": [ + "*" + ], + "android.text.method.MultiTapKeyListener": [ + "*" + ], + "android.text.method.NumberKeyListener": [ + "*" + ], + "android.text.method.QwertyKeyListener": [ + "*" + ], + "android.text.method.TextKeyListener": [ + "*" + ], + "android.text.method.TimeKeyListener": [ + "*" + ], + "android.transition.Transition$EpicenterCallback": [ + "*" + ], + "android.transition.Transition$TransitionListener": [ + "*" + ], + "android.view.ActionMode$Callback": [ + "*" + ], + "android.view.ActionProvider$VisibilityListener": [ + "*" + ], + "android.view.Choreographer$FrameCallback": [ + "*" + ], + "android.view.GestureDetector$OnContextClickListener": [ + "*" + ], + "android.view.GestureDetector$OnDoubleTapListener": [ + "*" + ], + "android.view.GestureDetector$OnGestureListener": [ + "*" + ], + "android.view.GestureDetector$SimpleOnGestureListener": [ + "*" + ], + "android.view.InputQueue$Callback": [ + "*" + ], + "android.view.KeyEvent$Callback": [ + "*" + ], + "android.view.MenuItem$OnActionExpandListener": [ + "*" + ], + "android.view.MenuItem$OnMenuItemClickListener": [ + "*" + ], + "android.view.OrientationEventListener": [ + "*" + ], + "android.view.OrientationListener": [ + "*" + ], + "android.view.PixelCopy$OnPixelCopyFinishedListener": [ + "*" + ], + "android.view.ScaleGestureDetector$OnScaleGestureListener": [ + "*" + ], + "android.view.ScaleGestureDetector$SimpleOnScaleGestureListener": [ + "*" + ], + "android.view.SurfaceHolder$Callback": [ + "*" + ], + "android.view.SurfaceHolder$Callback2": [ + "*" + ], + "android.view.TextureView$SurfaceTextureListener": [ + "*" + ], + "android.view.View$OnApplyWindowInsetsListener": [ + "*" + ], + "android.view.View$OnAttachStateChangeListener": [ + "*" + ], + "android.view.View$OnCapturedPointerListener": [ + "*" + ], + "android.view.View$OnContextClickListener": [ + "*" + ], + "android.view.View$OnCreateContextMenuListener": [ + "*" + ], + "android.view.View$OnDragListener": [ + "*" + ], + "android.view.View$OnFocusChangeListener": [ + "*" + ], + "android.view.View$OnGenericMotionListener": [ + "*" + ], + "android.view.View$OnHoverListener": [ + "*" + ], + "android.view.View$OnKeyListener": [ + "*" + ], + "android.view.View$OnLayoutChangeListener": [ + "*" + ], + "android.view.View$OnLongClickListener": [ + "*" + ], + "android.view.View$OnScrollChangeListener": [ + "*" + ], + "android.view.View$OnSystemUiVisibilityChangeListener": [ + "*" + ], + "android.view.View$OnTouchListener": [ + "*" + ], + "android.view.View$OnUnhandledKeyEventListener": [ + "*" + ], + "android.view.ViewGroup$OnHierarchyChangeListener": [ + "*" + ], + "android.view.ViewStub$OnInflateListener": [ + "*" + ], + "android.view.ViewTreeObserver$OnDrawListener": [ + "*" + ], + "android.view.ViewTreeObserver$OnGlobalFocusChangeListener": [ + "*" + ], + "android.view.ViewTreeObserver$OnGlobalLayoutListener": [ + "*" + ], + "android.view.ViewTreeObserver$OnPreDrawListener": [ + "*" + ], + "android.view.ViewTreeObserver$OnScrollChangedListener": [ + "*" + ], + "android.view.ViewTreeObserver$OnTouchModeChangeListener": [ + "*" + ], + "android.view.ViewTreeObserver$OnWindowAttachListener": [ + "*" + ], + "android.view.ViewTreeObserver$OnWindowFocusChangeListener": [ + "*" + ], + "android.view.Window$Callback": [ + "*" + ], + "android.view.Window$OnFrameMetricsAvailableListener": [ + "*" + ], + "android.view.Window$OnRestrictedCaptionAreaChangedListener": [ + "*" + ], + "android.view.accessibility.AccessibilityManager$AccessibilityStateChangeListener": [ + "*" + ], + "android.view.accessibility.AccessibilityManager$TouchExplorationStateChangeListener": [ + "*" + ], + "android.view.accessibility.CaptioningManager$CaptioningChangeListener": [ + "*" + ], + "android.view.animation.Animation$AnimationListener": [ + "*" + ], + "android.view.autofill.AutofillManager$AutofillCallback": [ + "*" + ], + "android.view.inputmethod.InputMethod$SessionCallback": [ + "*" + ], + "android.view.inputmethod.InputMethodSession$EventCallback": [ + "*" + ], + "android.view.textservice.SpellCheckerSession$SpellCheckerSessionListener": [ + "*" + ], + "android.webkit.DownloadListener": [ + "*" + ], + "android.webkit.GeolocationPermissions$Callback": [ + "*" + ], + "android.webkit.ValueCallback": [ + "*" + ], + "android.webkit.WebChromeClient$CustomViewCallback": [ + "*" + ], + "android.webkit.WebIconDatabase$IconListener": [ + "*" + ], + "android.webkit.WebMessagePort$WebMessageCallback": [ + "*" + ], + "android.webkit.WebView$FindListener": [ + "*" + ], + "android.webkit.WebView$PictureListener": [ + "*" + ], + "android.webkit.WebView$VisualStateCallback": [ + "*" + ], + "android.widget.AbsListView$MultiChoiceModeListener": [ + "*" + ], + "android.widget.AbsListView$OnScrollListener": [ + "*" + ], + "android.widget.AbsListView$RecyclerListener": [ + "*" + ], + "android.widget.ActionMenuView$OnMenuItemClickListener": [ + "*" + ], + "android.widget.AdapterView$OnItemClickListener": [ + "*" + ], + "android.widget.AdapterView$OnItemLongClickListener": [ + "*" + ], + "android.widget.AdapterView$OnItemSelectedListener": [ + "*" + ], + "android.widget.AdapterView.OnItemSelectedListener": [ + "*" + ], + "android.widget.AutoCompleteTextView$OnDismissListener": [ + "*" + ], + "android.widget.CalendarView$OnDateChangeListener": [ + "*" + ], + "android.widget.Chronometer$OnChronometerTickListener": [ + "*" + ], + "android.widget.CompoundButton$OnCheckedChangeListener": [ + "*" + ], + "android.widget.DatePicker$OnDateChangedListener": [ + "*" + ], + "android.widget.ExpandableListView$OnChildClickListener": [ + "*" + ], + "android.widget.ExpandableListView$OnGroupClickListener": [ + "*" + ], + "android.widget.ExpandableListView$OnGroupCollapseListener": [ + "*" + ], + "android.widget.ExpandableListView$OnGroupExpandListener": [ + "*" + ], + "android.widget.Filter$FilterListener": [ + "*" + ], + "android.widget.NumberPicker$OnDismissListener": [ + "*" + ], + "android.widget.NumberPicker$OnScrollListener": [ + "*" + ], + "android.widget.NumberPicker$OnValueChangeListener": [ + "*" + ], + "android.widget.PopupMenu$OnDismissListener": [ + "*" + ], + "android.widget.PopupMenu$OnMenuItemClickListener": [ + "*" + ], + "android.widget.PopupWindow$OnDismissListener": [ + "*" + ], + "android.widget.RadioGroup$OnCheckedChangeListener": [ + "*" + ], + "android.widget.RatingBar$OnRatingBarChangeListener": [ + "*" + ], + "android.widget.SearchView$OnCloseListener": [ + "*" + ], + "android.widget.SearchView$OnQueryTextListener": [ + "*" + ], + "android.widget.SearchView$OnSuggestionListener": [ + "*" + ], + "android.widget.SeekBar$OnSeekBarChangeListener": [ + "*" + ], + "android.widget.ShareActionProvider$OnShareTargetSelectedListener": [ + "*" + ], + "android.widget.SlidingDrawer$OnDrawerCloseListener": [ + "*" + ], + "android.widget.SlidingDrawer$OnDrawerOpenListener": [ + "*" + ], + "android.widget.SlidingDrawer$OnDrawerScrollListener": [ + "*" + ], + "android.widget.TabHost$OnTabChangeListener": [ + "*" + ], + "android.widget.TextView$OnEditorActionListener": [ + "*" + ], + "android.widget.TimePicker$OnTimeChangedListener": [ + "*" + ], + "android.widget.Toolbar$OnMenuItemClickListener": [ + "*" + ], + "android.widget.ZoomButtonsController$OnZoomListener": [ + "*" + ], + "com.android.volley.Response$ErrorListener": [ + "*" + ], + "com.android.volley.Response$Listener": [ + "*" + ], + "java.beans.PropertyChangeListener": [ + "*" + ], + "java.util.EventListener": [ + "*" + ], + "java.util.prefs.NodeChangeListener": [ + "*" + ], + "java.util.prefs.PreferenceChangeListener": [ + "*" + ], + "javax.net.ssl.HandshakeCompletedListener": [ + "*" + ], + "javax.net.ssl.SSLSessionBindingListener": [ + "*" + ], + "javax.security.auth.callback.Callback": [ + "*" + ], + "javax.security.auth.callback.PasswordCallback": [ + "*" + ], + "javax.sql.ConnectionEventListener": [ + "*" + ], + "javax.sql.RowSetListener": [ + "*" + ], + "javax.sql.StatementEventListener": [ + "*" + ], + "javax.xml.transform.ErrorListener": [ + "*" + ] + }, + // if CallBackEnhance is true,the following class's subclass are ignored,even if it's an inner class + "enhanceIgnore": [] + }, + Library: { + "Package": [ + "java.", + "sun.", + "javax.", + "com.sun.", + "com.google.", + "org.omg.", + "org.xml.", + "org.json.", + "org.w3c.dom", + "android.", + "androidx.", + "io.", + "kotlin.", + "kotlinx.", + "org.android.", + "org.apache.", + "okhttp3.", + "com.android.volley.", + "org.chromium.net.", + "junit.", + "jdk.", + "org.xmlpull.", + "dalvik." + ], + "ExcludeLibraryContains": [ + ] + }, + PointerFlowRule: { + "MethodName": { + "add": { + "params->@this": { + "I": [ + "p*" + ], + "O": [ + "@this" + ] + }, + "params->@this.data": { + "I": [ + "p*" + ], + "O": [ + "@this.data" + ] + } + }, + "addAll": { + "params->@this": { + "I": [ + "p*" + ], + "O": [ + "@this" + ] + }, + "@this.data->ret": { + "I": [ + "@this.data" + ], + "O": [ + "ret" + ] + } + }, + "putAll": { + "params->@this": { + "I": [ + "p*" + ], + "O": [ + "@this" + ] + }, + "params->@this.data": { + "I": [ + "p*" + ], + "O": [ + "@this.data" + ] + }, + "@this.data->ret": { + "I": [ + "@this.data" + ], + "O": [ + "ret" + ] + } + }, + "put": { + "params->@this": { + "I": [ + "p*" + ], + "O": [ + "@this" + ] + }, + "params->@this.data": { + "I": [ + "p*" + ], + "O": [ + "@this.data" + ] + }, + "@this.data->ret": { + "I": [ + "@this.data" + ], + "O": [ + "ret" + ] + } + }, + "iterator": { + "@this->ret": { + "I": [ + "@this" + ], + "O": [ + "ret" + ] + }, + "@this.data->ret": { + "I": [ + "@this.data" + ], + "O": [ + "ret" + ] + } + }, + "entrySet": { + "@this->ret": { + "I": [ + "@this" + ], + "O": [ + "ret" + ] + }, + "@this.data->ret": { + "I": [ + "@this.data" + ], + "O": [ + "ret" + ] + } + } + }, + "MethodSignature": { + "": { + "@this->ret": { + "I": [ + "@this" + ], + "O": [ + "ret" + ] + } + }, + "": { + "@this.data->ret": { + "I": [ + "@this.data" + ], + "O": [ + "ret" + ] + } + } + } + }, + VariableFlowRule: { + "InstantDefault": { + "params->ret": { + "I": [ + "p*" + ], + "O": [ + "ret" + ] + }, + "params->@this.data": { + "I": [ + "p*" + ], + "O": [ + "@this.data" + ] + }, + "params->@this": { + "I": [ + "p*" + ], + "O": [ + "@this" + ] + }, + "@this.data->ret": { + "I": [ + "@this.data" + ], + "O": [ + "ret" + ] + }, + "@this->ret": { + "I": [ + "@this" + ], + "O": [ + "ret" + ] + } + }, + "InstantSelfDefault": { + "params->ret": { + "I": [ + "p*" + ], + "O": [ + "ret" + ] + }, + "@this->ret": { + "I": [ + "@this" + ], + "O": [ + "ret" + ] + } + }, + "StaticDefault": { + "params->ret": { + "I": [ + "p*" + ], + "O": [ + "ret" + ] + } + }, + "MethodName": { + "checkPermission": {}, + "startActivityForResult": {}, + "getApplicationContext": {}, + "startActivity": {}, + "getContext": {}, + "": { + "params->@this.data": { + "I": [ + "p*" + ], + "O": [ + "@this.data" + ] + }, + "params->@this": { + "I": [ + "p*" + ], + "O": [ + "@this" + ] + } + }, + "equals": { + "params->@this": { + "I": [ + "p*" + ], + "O": [ + "@this" + ] + } + }, + "toString": { + "@this.data->ret": { + "I": [ + "@this.data" + ], + "O": [ + "ret" + ] + }, + "@this->ret": { + "I": [ + "@this" + ], + "O": [ + "ret" + ] + } + } + }, + "MethodSignature": { + "": {}, + "": { + "@this.data->ret": { + "I": [ + "@this.data" + ], + "O": [ + "ret" + ] + }, + "@this->ret": { + "I": [ + "@this" + ], + "O": [ + "ret" + ] + } + } + } + } +} \ No newline at end of file diff --git a/config/config.json5 b/config/config.json5 new file mode 100644 index 0000000..1905f70 --- /dev/null +++ b/config/config.json5 @@ -0,0 +1,10 @@ +{ + //apk to anlayze + "apkPath": "/Users/aaa/Downloads/app.apk", + //result output directory + "out": "out", + "rules": "unZipSlip.json", + "maxPointerAnalyzeTime": 600, + //print more info about this rule + "debugRule": "unZipSlip" +} diff --git a/config/rules/ContentProviderPathTraversal.json b/config/rules/ContentProviderPathTraversal.json new file mode 100644 index 0000000..baa63a2 --- /dev/null +++ b/config/rules/ContentProviderPathTraversal.json @@ -0,0 +1,57 @@ +{ + "ContentProviderPathTraversal": { + "SliceMode": true, + "traceDepth": 14, + "desc": { + "name": "ContentProviderPathTraversal", + "category": "", + "wiki": "", + "detail": "If the ContentProvider overwrites openFile but does not validate the Uri path, then an attacker may attempt to use ../ to access unexpected files", + "possibility": "", + "model": "" + }, + "source": { + "Param": { + "<*: android.os.ParcelFileDescriptor openFile(*)>": [ + "p0" + ] + } + }, + "sink": { + "": { + "TaintCheck": [ + "p0" + ] + } + }, + "sanitize": { + "getCanonicalFile": { + "": { + "TaintCheck": [ + "@this" + ] + }, + "": { + "TaintCheck": [ + "@this" + ] + } + }, + "containsDotDot": { + "": { + "TaintCheck": [ + "@this" + ], + "p0": [ + "..*" + ] + }, + "": { + "TaintCheck": [ + "@this" + ] + } + } + } + } +} \ No newline at end of file diff --git a/config/rules/PendingIntentMutable.json b/config/rules/PendingIntentMutable.json new file mode 100644 index 0000000..a655a80 --- /dev/null +++ b/config/rules/PendingIntentMutable.json @@ -0,0 +1,103 @@ +{ + "PendingIntentMutable": { + "SliceMode": true, + "traceDepth": 8, + "desc": { + "name": "PendingIntentMutable", + "category": "PendingIntent", + "detail": "When a malicious application received the vulnerable PendingIntent, the malicious application can modify the content of the original intent and the intent will be send under the context of the vulnerable application.", + "wiki": "", + "possibility": "4", + "model": "middle" + }, + "entry": { + }, + "source": { + "NewInstance": [ + "android.content.Intent" + ] + }, + "sanitize": { + "setClassNameString": { + "": { + "TaintCheck": [ + "@this" + ], + "NotTaint": [ + "p0", + "p1" + ] + } + }, + "initializeWithComponentAndUrl": { + "(java.lang.String,android.net.Uri,android.content.Context,java.lang.Class)>": { + "TaintCheck": [ + "@this" + ] + } + }, + "setClassName": { + "": { + "TaintCheck": [ + "@this" + ], + "NotTaint": [ + "p1" + ] + } + }, + "setComponent": { + "": { + "TaintCheck": [ + "@this" + ] + } + }, + "setPackage": { + "": { + "TaintCheck": [ + "@this" + ] + } + }, + "setClass": { + "": { + "TaintCheck": [ + "@this" + ], + "NotTaint": [ + "p1" + ] + } + }, + "initializeWithComponent": { + "(android.content.Context,java.lang.Class)>": { + "TaintCheck": [ + "@this" + ] + } + }, + "immutable": { + "": { + "p3": [ + "67108864:&" + ], + "TaintCheck": [ + "p2" + ] + } + } + }, + "sink": { + "": { + "TaintCheck": [ + "p2" + ], + "TaintParamType": [ + "android.content.Intent", + "android.content.Intent[]" + ] + } + } + } +} diff --git a/config/rules/broadcastIMEI.json b/config/rules/broadcastIMEI.json new file mode 100644 index 0000000..855a39b --- /dev/null +++ b/config/rules/broadcastIMEI.json @@ -0,0 +1,43 @@ +{ + "IMEI_SendBroadcast": { + "SliceMode": true, + "traceDepth": 12, + "PrimTypeAsTaint": true, + "desc": { + "name": "IMEI_SendBroadcast", + "detail": "IMEI SendBroadcast", + "category": "ComplianceInfo", + "complianceCategory": "IMEI_SendBroadcast", + "complianceCategoryDetail": "IMEI_SendBroadcast", + "level": "L4" + }, + "source": { + "Return": [ + "" + ] + }, + "sink": { + "<*: * sendBroadcast*(*)>": { + "LibraryOnly": true, + "TaintParamType": [ + "android.content.Intent", + "android.content.Intent[]" + ], + "TaintCheck": [ + "p*" + ] + }, + "<*: * sendOrderedBroadcast*(*)>": { + "LibraryOnly": true, + "TaintParamType": [ + "android.content.Intent", + "android.content.Intent[]" + ], + "TaintCheck": [ + "p*" + ] + } + } + } +} + diff --git a/config/rules/logSerial.json b/config/rules/logSerial.json new file mode 100644 index 0000000..8d601eb --- /dev/null +++ b/config/rules/logSerial.json @@ -0,0 +1,47 @@ +{ + "serial_Log": { + "SliceMode": true, + "traceDepth": 12, + "PrimTypeAsTaint": true, + "desc": { + "name": "serial_Log", + "detail": "write serial to log", + "category": "ComplianceInfo", + "complianceCategory": "serial_Log", + "complianceCategoryDetail": "serial_Log", + "level": "L4" + }, + "source": { + "Field": [ + "" + ] + }, + "sink": { + "": { + "TaintCheck": [ + "p*" + ] + }, + "": { + "TaintCheck": [ + "p*" + ] + }, + "": { + "TaintCheck": [ + "p*" + ] + }, + "": { + "TaintCheck": [ + "p*" + ] + }, + "": { + "TaintCheck": [ + "p*" + ] + } + } + } +} \ No newline at end of file diff --git a/config/rules/unZipSlip.json b/config/rules/unZipSlip.json new file mode 100644 index 0000000..c69533d --- /dev/null +++ b/config/rules/unZipSlip.json @@ -0,0 +1,62 @@ +{ + "unZipSlip": { + "SliceMode": true, + "traceDepth": 8, + "desc": { + "name": "unZipSlip", + "category": "FileRisk", + "detail": "ZIP Slip is a highly critical security vulnerability aimed at these kind of applications. ZIP Slip makes your application vulnerable to Path traversal attack and Sensitive data exposure.", + "wiki": "", + "possibility": "4", + "model": "middle" + }, + "entry": { + }, + "source": { + "Return": [ + "" + ] + }, + "sanitize": { + "rule1": { + "": { + "TaintCheck": [ + "@this" + ] + } + }, + "containsDotDot": { + "": { + "TaintCheck": [ + "@this" + ], + "p0": [ + "..*" + ] + } + }, + "indexDotDot": { + "": { + "TaintCheck": [ + "@this" + ], + "p0": [ + "..*" + ] + } + } + }, + "sink": { + "(*)>": { + "TaintCheck": [ + "p*" + ] + }, + "(*)>": { + "TaintCheck": [ + "p*" + ] + } + } + } +} \ No newline at end of file diff --git a/config/tools/ApkName.sh b/config/tools/ApkName.sh new file mode 100755 index 0000000..b07bd73 --- /dev/null +++ b/config/tools/ApkName.sh @@ -0,0 +1,15 @@ +#!/bin/bash +apk=$1 + +name=`aapt dump badging $apk | grep "application-label-zh-CN"|awk -F ":" '{print $2}' | awk -F "'" '{print $2}'` + +if [ "$name" == "" ] +then + name=`aapt dump badging $apk | grep "application-label-zh"|awk -F ":" '{print $2}' | awk -F "'" '{print $2}'` + if [ "$name" == "" ] + then + name=`aapt dump badging $apk | grep "application-label"|awk -F ":" '{print $2}' | awk -F "'" '{print $2}'` + fi +fi + +echo $name diff --git a/config/tools/jadx/LICENSE b/config/tools/jadx/LICENSE new file mode 100644 index 0000000..8dada3e --- /dev/null +++ b/config/tools/jadx/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "{}" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright {yyyy} {name of copyright owner} + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/config/tools/jadx/NOTICE b/config/tools/jadx/NOTICE new file mode 100644 index 0000000..5c0b69a --- /dev/null +++ b/config/tools/jadx/NOTICE @@ -0,0 +1,213 @@ +The majority of jadx is written and copyrighted by me (Skylot) +and released under the Apache 2.0 license (see LICENSE file for full license text): + +******************************************************************************* +Copyright 2015, Skylot + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +******************************************************************************* + + +Various portions of the code including dx library are taken from +the Android Open Source Project, and are used in accordance with +the following license: + +******************************************************************************* +Copyright (C) 2007 The Android Open Source Project + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +******************************************************************************* + + +Other binary libraries used in 'jadx' +===================================== + +JCommander library (http://jcommander.org/) released under the following license: + +******************************************************************************* +Copyright 2012, Cedric Beust + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +******************************************************************************* + + +SLF4J source code and binaries are distributed under the following license: + +******************************************************************************* +Copyright (c) 2004-2011 QOS.ch + All rights reserved. + + Permission is hereby granted, free of charge, to any person obtaining + a copy of this software and associated documentation files (the + "Software"), to deal in the Software without restriction, including + without limitation the rights to use, copy, modify, merge, publish, + distribute, sublicense, and/or sell copies of the Software, and to + permit persons to whom the Software is furnished to do so, subject to + the following conditions: + + The above copyright notice and this permission notice shall be + included in all copies or substantial portions of the Software. + + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, + EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF + MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND + NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE + LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION + OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION + WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +******************************************************************************* + + +Logback source code and binaries are dual-licensed under the EPL v1.0 and the LGPL 2.1, or more formally: + +******************************************************************************* +Logback: the reliable, generic, fast and flexible logging framework. +Copyright (C) 1999-2012, QOS.ch. All rights reserved. + +This program and the accompanying materials are dual-licensed under +either the terms of the Eclipse Public License v1.0 as published by +the Eclipse Foundation + + or (per the licensee's choosing) + +under the terms of the GNU Lesser General Public License version 2.1 +as published by the Free Software Foundation. +******************************************************************************* + + +ASM library: + +******************************************************************************* +Copyright (c) 2000-2011 INRIA, France Telecom +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: + +1. Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + +2. Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + +3. Neither the name of the copyright holders nor the names of its + contributors may be used to endorse or promote products derived from + this software without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE +LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF +THE POSSIBILITY OF SUCH DAMAGE. +******************************************************************************* + + + +Jadx-gui components +=================== + +RSyntaxTextArea library (https://github.com/bobbylight/RSyntaxTextArea) +licensed under modified BSD license: + +******************************************************************************* +Copyright (c) 2012, Robert Futrell +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions are met: + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + * Neither the name of the author nor the names of its contributors may + be used to endorse or promote products derived from this software + without specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL BE LIABLE FOR ANY +DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND +ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +******************************************************************************* + + +Concurrent Trees (https://code.google.com/p/concurrent-trees/) +licenced under Apache License 2.0: + +******************************************************************************* +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. +******************************************************************************* + + +Image Viewer (https://github.com/kazocsaba/imageviewer) + +******************************************************************************* +Copyright (c) 2008-2012 Kazó Csaba + +Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +******************************************************************************* + +JFontChooser Component - http://sourceforge.jp/projects/jfontchooser/ + +Icons copied from several places: + - Eclipse Project (JDT UI) - licensed under EPL v1.0 (http://www.eclipse.org/legal/epl-v10.html) + - famfamfam silk icon set (http://www.famfamfam.com/lab/icons/silk/) - licensed + under Creative Commons Attribution 2.5 License (http://creativecommons.org/licenses/by/2.5/) diff --git a/config/tools/jadx/README.md b/config/tools/jadx/README.md new file mode 100644 index 0000000..9d4f431 --- /dev/null +++ b/config/tools/jadx/README.md @@ -0,0 +1,158 @@ + + +## JADX + +[![Build status](https://github.com/skylot/jadx/workflows/Build/badge.svg)](https://github.com/skylot/jadx/actions?query=workflow%3ABuild) +[![Alerts from lgtm.com](https://img.shields.io/lgtm/alerts/g/skylot/jadx.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/skylot/jadx/alerts/) +[![semantic-release](https://img.shields.io/badge/%20%20%F0%9F%93%A6%F0%9F%9A%80-semantic--release-e10079.svg)](https://github.com/semantic-release/semantic-release) +[![Maven Central](https://img.shields.io/maven-central/v/io.github.skylot/jadx-core)](https://search.maven.org/search?q=g:io.github.skylot%20AND%20jadx) +[![License](http://img.shields.io/:license-apache-blue.svg)](http://www.apache.org/licenses/LICENSE-2.0.html) + +**jadx** - Dex to Java decompiler + +Command line and GUI tools for producing Java source code from Android Dex and Apk files + +:exclamation::exclamation::exclamation: Please note that in most cases **jadx** can't decompile all 100% of the code, so errors will occur. Check [Troubleshooting guide](https://github.com/skylot/jadx/wiki/Troubleshooting-Q&A#decompilation-issues) for workarounds + +**Main features:** +- decompile Dalvik bytecode to java classes from APK, dex, aar, aab and zip files +- decode `AndroidManifest.xml` and other resources from `resources.arsc` +- deobfuscator included + +**jadx-gui features:** +- view decompiled code with highlighted syntax +- jump to declaration +- find usage +- full text search +- smali debugger, check [wiki page](https://github.com/skylot/jadx/wiki/Smali-debugger) for setup and usage + +Jadx-gui key bindings can be found [here](https://github.com/skylot/jadx/wiki/JADX-GUI-Key-bindings) + +See these features in action here: [jadx-gui features overview](https://github.com/skylot/jadx/wiki/jadx-gui-features-overview) + + + +### Download +- release from [github: ![Latest release](https://img.shields.io/github/release/skylot/jadx.svg)](https://github.com/skylot/jadx/releases/latest) +- latest [unstable build](https://nightly.link/skylot/jadx/workflows/build/master) + +After download unpack zip file go to `bin` directory and run: +- `jadx` - command line version +- `jadx-gui` - UI version + +On Windows run `.bat` files with double-click\ +**Note:** ensure you have installed Java 11 or later 64-bit version. +For Windows, you can download it from [oracle.com](https://www.oracle.com/java/technologies/downloads/#jdk17-windows) (select x64 Installer). + +### Install +1. Arch linux + ```bash + sudo pacman -S jadx + ``` +2. macOS + ```bash + brew install jadx + ``` + +### Use jadx as a library +You can use jadx in your java projects, check details on [wiki page](https://github.com/skylot/jadx/wiki/Use-jadx-as-a-library) + +### Build from source +JDK 8 or higher must be installed: +``` +git clone https://github.com/skylot/jadx.git +cd jadx +./gradlew dist +``` + +(on Windows, use `gradlew.bat` instead of `./gradlew`) + +Scripts for run jadx will be placed in `build/jadx/bin` +and also packed to `build/jadx-.zip` + +### Usage +``` +jadx[-gui] [options] (.apk, .dex, .jar, .class, .smali, .zip, .aar, .arsc, .aab) +options: + -d, --output-dir - output directory + -ds, --output-dir-src - output directory for sources + -dr, --output-dir-res - output directory for resources + -r, --no-res - do not decode resources + -s, --no-src - do not decompile source code + --single-class - decompile a single class, full name, raw or alias + --single-class-output - file or dir for write if decompile a single class + --output-format - can be 'java' or 'json', default: java + -e, --export-gradle - save as android gradle project + -j, --threads-count - processing threads count, default: 4 + -m, --decompilation-mode - code output mode: + 'auto' - trying best options (default) + 'restructure' - restore code structure (normal java code) + 'simple' - simplified instructions (linear, with goto's) + 'fallback' - raw instructions without modifications + --show-bad-code - show inconsistent code (incorrectly decompiled) + --no-imports - disable use of imports, always write entire package name + --no-debug-info - disable debug info + --add-debug-lines - add comments with debug line numbers if available + --no-inline-anonymous - disable anonymous classes inline + --no-inline-methods - disable methods inline + --no-replace-consts - don't replace constant value with matching constant field + --escape-unicode - escape non latin characters in strings (with \u) + --respect-bytecode-access-modifiers - don't change original access modifiers + --deobf - activate deobfuscation + --deobf-min - min length of name, renamed if shorter, default: 3 + --deobf-max - max length of name, renamed if longer, default: 64 + --deobf-cfg-file - deobfuscation map file, default: same dir and name as input file with '.jobf' extension + --deobf-cfg-file-mode - set mode for handle deobfuscation map file: + 'read' - read if found, don't save (default) + 'read-or-save' - read if found, save otherwise (don't overwrite) + 'overwrite' - don't read, always save + 'ignore' - don't read and don't save + --deobf-rewrite-cfg - set '--deobf-cfg-file-mode' to 'overwrite' (deprecated) + --deobf-use-sourcename - use source file name as class name alias + --deobf-parse-kotlin-metadata - parse kotlin metadata to class and package names + --use-kotlin-methods-for-var-names - use kotlin intrinsic methods to rename variables, values: disable, apply, apply-and-hide, default: apply + --rename-flags - fix options (comma-separated list of): + 'case' - fix case sensitivity issues (according to --fs-case-sensitive option), + 'valid' - rename java identifiers to make them valid, + 'printable' - remove non-printable chars from identifiers, + or single 'none' - to disable all renames + or single 'all' - to enable all (default) + --fs-case-sensitive - treat filesystem as case sensitive, false by default + --cfg - save methods control flow graph to dot file + --raw-cfg - save methods control flow graph (use raw instructions) + -f, --fallback - set '--decompilation-mode' to 'fallback' (deprecated) + --use-dx - use dx/d8 to convert java bytecode + --comments-level - set code comments level, values: error, warn, info, debug, user-only, none, default: info + --log-level - set log level, values: quiet, progress, error, warn, info, debug, default: progress + -v, --verbose - verbose output (set --log-level to DEBUG) + -q, --quiet - turn off output (set --log-level to QUIET) + --version - print jadx version + -h, --help - print this help + +Plugin options (-P=): + 1) dex-input (Load .dex and .apk files) + -Pdex-input.verify-checksum - Verify dex file checksum before load, values: [yes, no], default: yes + 2) java-convert (Convert .jar and .class files to dex) + -Pjava-convert.mode - Convert mode, values: [dx, d8, both], default: both + -Pjava-convert.d8-desugar - Use desugar in d8, values: [yes, no], default: no + +Examples: + jadx -d out classes.dex + jadx --rename-flags "none" classes.dex + jadx --rename-flags "valid, printable" classes.dex + jadx --log-level ERROR app.apk + jadx -Pdex-input.verify-checksum=no app.apk +``` +These options also worked on jadx-gui running from command line and override options from preferences dialog + +### Troubleshooting +Please check wiki page [Troubleshooting Q&A](https://github.com/skylot/jadx/wiki/Troubleshooting-Q&A) + +### Contributing +To support this project you can: + - Post thoughts about new features/optimizations that important to you + - Submit decompilation issues, please read before proceed: [Open issue](CONTRIBUTING.md#Open-Issue) + - Open pull request, please follow these rules: [Pull Request Process](CONTRIBUTING.md#Pull-Request-Process) + +--------------------------------------- +*Licensed under the Apache 2.0 License* diff --git a/config/tools/jadx/bin/jadx b/config/tools/jadx/bin/jadx new file mode 100755 index 0000000..fd0ea1f --- /dev/null +++ b/config/tools/jadx/bin/jadx @@ -0,0 +1,233 @@ +#!/bin/sh + +# +# Copyright © 2015-2021 the original authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +############################################################################## +# +# jadx start up script for POSIX generated by Gradle. +# +# Important for running: +# +# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is +# noncompliant, but you have some other compliant shell such as ksh or +# bash, then to run this script, type that shell name before the whole +# command line, like: +# +# ksh jadx +# +# Busybox and similar reduced shells will NOT work, because this script +# requires all of these POSIX shell features: +# * functions; +# * expansions «$var», «${var}», «${var:-default}», «${var+SET}», +# «${var#prefix}», «${var%suffix}», and «$( cmd )»; +# * compound commands having a testable exit status, especially «case»; +# * various built-in commands including «command», «set», and «ulimit». +# +# Important for patching: +# +# (2) This script targets any POSIX shell, so it avoids extensions provided +# by Bash, Ksh, etc; in particular arrays are avoided. +# +# The "traditional" practice of packing multiple parameters into a +# space-separated string is a well documented source of bugs and security +# problems, so this is (mostly) avoided, by progressively accumulating +# options in "$@", and eventually passing that to Java. +# +# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS, +# and JADX_OPTS) rely on word-splitting, this is performed explicitly; +# see the in-line comments for details. +# +# There are tweaks for specific operating systems such as AIX, CygWin, +# Darwin, MinGW, and NonStop. +# +# (3) This script is generated from the Groovy template +# https://github.com/gradle/gradle/blob/master/subprojects/plugins/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt +# within the Gradle project. +# +# You can find Gradle at https://github.com/gradle/gradle/. +# +############################################################################## + +# Attempt to set APP_HOME + +# Resolve links: $0 may be a link +app_path=$0 + +# Need this for daisy-chained symlinks. +while + APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path + [ -h "$app_path" ] +do + ls=$( ls -ld "$app_path" ) + link=${ls#*' -> '} + case $link in #( + /*) app_path=$link ;; #( + *) app_path=$APP_HOME$link ;; + esac +done + +APP_HOME=$( cd "${APP_HOME:-./}.." && pwd -P ) || exit + +APP_NAME="jadx" +APP_BASE_NAME=${0##*/} + +# Add default JVM options here. You can also use JAVA_OPTS and JADX_OPTS to pass JVM options to this script. +DEFAULT_JVM_OPTS='"-Xms128M" "-XX:MaxRAMPercentage=70.0" "-XX:+UseG1GC"' + +# Use the maximum available, or set MAX_FD != -1 to use that value. +MAX_FD=maximum + +warn () { + echo "$*" +} >&2 + +die () { + echo + echo "$*" + echo + exit 1 +} >&2 + +# OS specific support (must be 'true' or 'false'). +cygwin=false +msys=false +darwin=false +nonstop=false +case "$( uname )" in #( + CYGWIN* ) cygwin=true ;; #( + Darwin* ) darwin=true ;; #( + MSYS* | MINGW* ) msys=true ;; #( + NONSTOP* ) nonstop=true ;; +esac + +CLASSPATH=$APP_HOME/lib/jadx-cli-1.3.5.jar:$APP_HOME/lib/jadx-core-1.3.5.jar:$APP_HOME/lib/logback-classic-1.2.11.jar:$APP_HOME/lib/jadx-java-convert-1.3.5.jar:$APP_HOME/lib/jadx-smali-input-1.3.5.jar:$APP_HOME/lib/jadx-dex-input-1.3.5.jar:$APP_HOME/lib/jadx-java-input-1.3.5.jar:$APP_HOME/lib/jadx-plugins-api-1.3.5.jar:$APP_HOME/lib/raung-disasm-0.0.2.jar:$APP_HOME/lib/raung-common-0.0.2.jar:$APP_HOME/lib/slf4j-api-1.7.36.jar:$APP_HOME/lib/baksmali-2.5.2.jar:$APP_HOME/lib/smali-2.5.2.jar:$APP_HOME/lib/util-2.5.2.jar:$APP_HOME/lib/jcommander-1.82.jar:$APP_HOME/lib/gson-2.9.0.jar:$APP_HOME/lib/aapt2-proto-4.2.1-7147631.jar:$APP_HOME/lib/protobuf-java-3.11.4.jar:$APP_HOME/lib/logback-core-1.2.11.jar:$APP_HOME/lib/dexlib2-2.5.2.jar:$APP_HOME/lib/guava-30.1.1-jre.jar:$APP_HOME/lib/dalvik-dx-11.0.0_r3.jar:$APP_HOME/lib/r8-3.3.28.jar:$APP_HOME/lib/asm-9.3.jar:$APP_HOME/lib/antlr-3.5.2.jar:$APP_HOME/lib/ST4-4.0.8.jar:$APP_HOME/lib/antlr-runtime-3.5.2.jar:$APP_HOME/lib/stringtemplate-3.2.1.jar:$APP_HOME/lib/jsr305-3.0.2.jar:$APP_HOME/lib/failureaccess-1.0.1.jar:$APP_HOME/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:$APP_HOME/lib/checker-qual-3.8.0.jar:$APP_HOME/lib/error_prone_annotations-2.5.1.jar:$APP_HOME/lib/j2objc-annotations-1.3.jar:$APP_HOME/lib/antlr-2.7.7.jar + + +# Determine the Java command to use to start the JVM. +if [ -n "$JAVA_HOME" ] ; then + if [ -x "$JAVA_HOME/jre/sh/java" ] ; then + # IBM's JDK on AIX uses strange locations for the executables + JAVACMD=$JAVA_HOME/jre/sh/java + else + JAVACMD=$JAVA_HOME/bin/java + fi + if [ ! -x "$JAVACMD" ] ; then + die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +else + JAVACMD=java + which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." +fi + +# Increase the maximum file descriptors if we can. +if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then + case $MAX_FD in #( + max*) + MAX_FD=$( ulimit -H -n ) || + warn "Could not query maximum file descriptor limit" + esac + case $MAX_FD in #( + '' | soft) :;; #( + *) + ulimit -n "$MAX_FD" || + warn "Could not set maximum file descriptor limit to $MAX_FD" + esac +fi + +# Collect all arguments for the java command, stacking in reverse order: +# * args from the command line +# * the main class name +# * -classpath +# * -D...appname settings +# * --module-path (only if needed) +# * DEFAULT_JVM_OPTS, JAVA_OPTS, and JADX_OPTS environment variables. + +# For Cygwin or MSYS, switch paths to Windows format before running java +if "$cygwin" || "$msys" ; then + APP_HOME=$( cygpath --path --mixed "$APP_HOME" ) + CLASSPATH=$( cygpath --path --mixed "$CLASSPATH" ) + + JAVACMD=$( cygpath --unix "$JAVACMD" ) + + # Now convert the arguments - kludge to limit ourselves to /bin/sh + for arg do + if + case $arg in #( + -*) false ;; # don't mess with options #( + /?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath + [ -e "$t" ] ;; #( + *) false ;; + esac + then + arg=$( cygpath --path --ignore --mixed "$arg" ) + fi + # Roll the args list around exactly as many times as the number of + # args, so each arg winds up back in the position where it started, but + # possibly modified. + # + # NB: a `for` loop captures its iteration list before it begins, so + # changing the positional parameters here affects neither the number of + # iterations, nor the values presented in `arg`. + shift # remove old arg + set -- "$@" "$arg" # push replacement arg + done +fi + +# Collect all arguments for the java command; +# * $DEFAULT_JVM_OPTS, $JAVA_OPTS, and $JADX_OPTS can contain fragments of +# shell script including quotes and variable substitutions, so put them in +# double quotes to make sure that they get re-expanded; and +# * put everything else in single quotes, so that it's not re-expanded. + +set -- \ + -classpath "$CLASSPATH" \ + jadx.cli.JadxCLI \ + "$@" + +# Use "xargs" to parse quoted args. +# +# With -n1 it outputs one arg per line, with the quotes and backslashes removed. +# +# In Bash we could simply go: +# +# readarray ARGS < <( xargs -n1 <<<"$var" ) && +# set -- "${ARGS[@]}" "$@" +# +# but POSIX shell has neither arrays nor command substitution, so instead we +# post-process each arg (as a line of input to sed) to backslash-escape any +# character that might be a shell metacharacter, then use eval to reverse +# that process (while maintaining the separation between arguments), and wrap +# the whole thing up as a single "set" statement. +# +# This will of course break if any of these variables contains a newline or +# an unmatched quote. +# + +eval "set -- $( + printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $JADX_OPTS" | + xargs -n1 | + sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' | + tr '\n' ' ' + )" '"$@"' + +exec "$JAVACMD" "$@" diff --git a/config/tools/jadx/bin/jadx-gui b/config/tools/jadx/bin/jadx-gui new file mode 100755 index 0000000..f25ba84 --- /dev/null +++ b/config/tools/jadx/bin/jadx-gui @@ -0,0 +1,233 @@ +#!/bin/sh + +# +# Copyright © 2015-2021 the original authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +############################################################################## +# +# jadx-gui start up script for POSIX generated by Gradle. +# +# Important for running: +# +# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is +# noncompliant, but you have some other compliant shell such as ksh or +# bash, then to run this script, type that shell name before the whole +# command line, like: +# +# ksh jadx-gui +# +# Busybox and similar reduced shells will NOT work, because this script +# requires all of these POSIX shell features: +# * functions; +# * expansions «$var», «${var}», «${var:-default}», «${var+SET}», +# «${var#prefix}», «${var%suffix}», and «$( cmd )»; +# * compound commands having a testable exit status, especially «case»; +# * various built-in commands including «command», «set», and «ulimit». +# +# Important for patching: +# +# (2) This script targets any POSIX shell, so it avoids extensions provided +# by Bash, Ksh, etc; in particular arrays are avoided. +# +# The "traditional" practice of packing multiple parameters into a +# space-separated string is a well documented source of bugs and security +# problems, so this is (mostly) avoided, by progressively accumulating +# options in "$@", and eventually passing that to Java. +# +# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS, +# and JADX_GUI_OPTS) rely on word-splitting, this is performed explicitly; +# see the in-line comments for details. +# +# There are tweaks for specific operating systems such as AIX, CygWin, +# Darwin, MinGW, and NonStop. +# +# (3) This script is generated from the Groovy template +# https://github.com/gradle/gradle/blob/master/subprojects/plugins/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt +# within the Gradle project. +# +# You can find Gradle at https://github.com/gradle/gradle/. +# +############################################################################## + +# Attempt to set APP_HOME + +# Resolve links: $0 may be a link +app_path=$0 + +# Need this for daisy-chained symlinks. +while + APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path + [ -h "$app_path" ] +do + ls=$( ls -ld "$app_path" ) + link=${ls#*' -> '} + case $link in #( + /*) app_path=$link ;; #( + *) app_path=$APP_HOME$link ;; + esac +done + +APP_HOME=$( cd "${APP_HOME:-./}.." && pwd -P ) || exit + +APP_NAME="jadx-gui" +APP_BASE_NAME=${0##*/} + +# Add default JVM options here. You can also use JAVA_OPTS and JADX_GUI_OPTS to pass JVM options to this script. +DEFAULT_JVM_OPTS='"-Xms128M" "-XX:MaxRAMPercentage=70.0" "-XX:+UseG1GC" "-Dawt.useSystemAAFontSettings=lcd" "-Dswing.aatext=true"' + +# Use the maximum available, or set MAX_FD != -1 to use that value. +MAX_FD=maximum + +warn () { + echo "$*" +} >&2 + +die () { + echo + echo "$*" + echo + exit 1 +} >&2 + +# OS specific support (must be 'true' or 'false'). +cygwin=false +msys=false +darwin=false +nonstop=false +case "$( uname )" in #( + CYGWIN* ) cygwin=true ;; #( + Darwin* ) darwin=true ;; #( + MSYS* | MINGW* ) msys=true ;; #( + NONSTOP* ) nonstop=true ;; +esac + +CLASSPATH=$APP_HOME/lib/jadx-gui-1.3.5.jar:$APP_HOME/lib/jfontchooser-1.0.5.jar:$APP_HOME/lib/jadx-cli-1.3.5.jar:$APP_HOME/lib/jadx-core-1.3.5.jar:$APP_HOME/lib/logback-classic-1.2.11.jar:$APP_HOME/lib/jadx-java-convert-1.3.5.jar:$APP_HOME/lib/jadx-smali-input-1.3.5.jar:$APP_HOME/lib/jadx-dex-input-1.3.5.jar:$APP_HOME/lib/jadx-java-input-1.3.5.jar:$APP_HOME/lib/jadx-plugins-api-1.3.5.jar:$APP_HOME/lib/raung-disasm-0.0.2.jar:$APP_HOME/lib/raung-common-0.0.2.jar:$APP_HOME/lib/slf4j-api-1.7.36.jar:$APP_HOME/lib/baksmali-2.5.2.jar:$APP_HOME/lib/smali-2.5.2.jar:$APP_HOME/lib/util-2.5.2.jar:$APP_HOME/lib/jcommander-1.82.jar:$APP_HOME/lib/rsyntaxtextarea-3.2.0.jar:$APP_HOME/lib/image-viewer-1.2.3.jar:$APP_HOME/lib/flatlaf-intellij-themes-2.1.jar:$APP_HOME/lib/flatlaf-extras-2.1.jar:$APP_HOME/lib/flatlaf-2.1.jar:$APP_HOME/lib/svgSalamander-1.1.3.jar:$APP_HOME/lib/gson-2.9.0.jar:$APP_HOME/lib/commons-text-1.9.jar:$APP_HOME/lib/commons-lang3-3.12.0.jar:$APP_HOME/lib/rxjava2-swing-0.3.7.jar:$APP_HOME/lib/rxjava-2.2.21.jar:$APP_HOME/lib/apksig-4.2.1.jar:$APP_HOME/lib/jdwp-1.0.jar:$APP_HOME/lib/aapt2-proto-4.2.1-7147631.jar:$APP_HOME/lib/protobuf-java-3.11.4.jar:$APP_HOME/lib/logback-core-1.2.11.jar:$APP_HOME/lib/reactive-streams-1.0.3.jar:$APP_HOME/lib/dexlib2-2.5.2.jar:$APP_HOME/lib/guava-30.1.1-jre.jar:$APP_HOME/lib/dalvik-dx-11.0.0_r3.jar:$APP_HOME/lib/r8-3.3.28.jar:$APP_HOME/lib/asm-9.3.jar:$APP_HOME/lib/antlr-3.5.2.jar:$APP_HOME/lib/ST4-4.0.8.jar:$APP_HOME/lib/antlr-runtime-3.5.2.jar:$APP_HOME/lib/stringtemplate-3.2.1.jar:$APP_HOME/lib/jsr305-3.0.2.jar:$APP_HOME/lib/failureaccess-1.0.1.jar:$APP_HOME/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:$APP_HOME/lib/checker-qual-3.8.0.jar:$APP_HOME/lib/error_prone_annotations-2.5.1.jar:$APP_HOME/lib/j2objc-annotations-1.3.jar:$APP_HOME/lib/antlr-2.7.7.jar + + +# Determine the Java command to use to start the JVM. +if [ -n "$JAVA_HOME" ] ; then + if [ -x "$JAVA_HOME/jre/sh/java" ] ; then + # IBM's JDK on AIX uses strange locations for the executables + JAVACMD=$JAVA_HOME/jre/sh/java + else + JAVACMD=$JAVA_HOME/bin/java + fi + if [ ! -x "$JAVACMD" ] ; then + die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +else + JAVACMD=java + which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." +fi + +# Increase the maximum file descriptors if we can. +if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then + case $MAX_FD in #( + max*) + MAX_FD=$( ulimit -H -n ) || + warn "Could not query maximum file descriptor limit" + esac + case $MAX_FD in #( + '' | soft) :;; #( + *) + ulimit -n "$MAX_FD" || + warn "Could not set maximum file descriptor limit to $MAX_FD" + esac +fi + +# Collect all arguments for the java command, stacking in reverse order: +# * args from the command line +# * the main class name +# * -classpath +# * -D...appname settings +# * --module-path (only if needed) +# * DEFAULT_JVM_OPTS, JAVA_OPTS, and JADX_GUI_OPTS environment variables. + +# For Cygwin or MSYS, switch paths to Windows format before running java +if "$cygwin" || "$msys" ; then + APP_HOME=$( cygpath --path --mixed "$APP_HOME" ) + CLASSPATH=$( cygpath --path --mixed "$CLASSPATH" ) + + JAVACMD=$( cygpath --unix "$JAVACMD" ) + + # Now convert the arguments - kludge to limit ourselves to /bin/sh + for arg do + if + case $arg in #( + -*) false ;; # don't mess with options #( + /?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath + [ -e "$t" ] ;; #( + *) false ;; + esac + then + arg=$( cygpath --path --ignore --mixed "$arg" ) + fi + # Roll the args list around exactly as many times as the number of + # args, so each arg winds up back in the position where it started, but + # possibly modified. + # + # NB: a `for` loop captures its iteration list before it begins, so + # changing the positional parameters here affects neither the number of + # iterations, nor the values presented in `arg`. + shift # remove old arg + set -- "$@" "$arg" # push replacement arg + done +fi + +# Collect all arguments for the java command; +# * $DEFAULT_JVM_OPTS, $JAVA_OPTS, and $JADX_GUI_OPTS can contain fragments of +# shell script including quotes and variable substitutions, so put them in +# double quotes to make sure that they get re-expanded; and +# * put everything else in single quotes, so that it's not re-expanded. + +set -- \ + -classpath "$CLASSPATH" \ + jadx.gui.JadxGUI \ + "$@" + +# Use "xargs" to parse quoted args. +# +# With -n1 it outputs one arg per line, with the quotes and backslashes removed. +# +# In Bash we could simply go: +# +# readarray ARGS < <( xargs -n1 <<<"$var" ) && +# set -- "${ARGS[@]}" "$@" +# +# but POSIX shell has neither arrays nor command substitution, so instead we +# post-process each arg (as a line of input to sed) to backslash-escape any +# character that might be a shell metacharacter, then use eval to reverse +# that process (while maintaining the separation between arguments), and wrap +# the whole thing up as a single "set" statement. +# +# This will of course break if any of these variables contains a newline or +# an unmatched quote. +# + +eval "set -- $( + printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $JADX_GUI_OPTS" | + xargs -n1 | + sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' | + tr '\n' ' ' + )" '"$@"' + +exec "$JAVACMD" "$@" diff --git a/config/tools/jadx/bin/jadx-gui.bat b/config/tools/jadx/bin/jadx-gui.bat new file mode 100755 index 0000000..07d4a41 --- /dev/null +++ b/config/tools/jadx/bin/jadx-gui.bat @@ -0,0 +1,89 @@ +@rem +@rem Copyright 2015 the original author or authors. +@rem +@rem Licensed under the Apache License, Version 2.0 (the "License"); +@rem you may not use this file except in compliance with the License. +@rem You may obtain a copy of the License at +@rem +@rem https://www.apache.org/licenses/LICENSE-2.0 +@rem +@rem Unless required by applicable law or agreed to in writing, software +@rem distributed under the License is distributed on an "AS IS" BASIS, +@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +@rem See the License for the specific language governing permissions and +@rem limitations under the License. +@rem + +@if "%DEBUG%" == "" @echo off +@rem ########################################################################## +@rem +@rem jadx-gui startup script for Windows +@rem +@rem ########################################################################## + +@rem Set local scope for the variables with windows NT shell +if "%OS%"=="Windows_NT" setlocal + +set DIRNAME=%~dp0 +if "%DIRNAME%" == "" set DIRNAME=. +set APP_BASE_NAME=%~n0 +set APP_HOME=%DIRNAME%.. + +@rem Resolve any "." and ".." in APP_HOME to make it shorter. +for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi + +@rem Add default JVM options here. You can also use JAVA_OPTS and JADX_GUI_OPTS to pass JVM options to this script. +set DEFAULT_JVM_OPTS="-Xms128M" "-XX:MaxRAMPercentage=70.0" "-XX:+UseG1GC" "-Dawt.useSystemAAFontSettings=lcd" "-Dswing.aatext=true" + +@rem Find javaw.exe +if defined JAVA_HOME goto findJavaFromJavaHome + +set JAVA_EXE=javaw.exe +%JAVA_EXE% -version >NUL 2>&1 +if "%ERRORLEVEL%" == "0" goto execute + +echo. +echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. +echo. +echo Please set the JAVA_HOME variable in your environment to match the +echo location of your Java installation. + +goto fail + +:findJavaFromJavaHome +set JAVA_HOME=%JAVA_HOME:"=% +set JAVA_EXE=%JAVA_HOME%/bin/javaw.exe + +if exist "%JAVA_EXE%" goto execute + +echo. +echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% +echo. +echo Please set the JAVA_HOME variable in your environment to match the +echo location of your Java installation. + +goto fail + +:execute +@rem Setup the command line + +set CLASSPATH=%APP_HOME%\lib\jadx-gui-1.3.5.jar;%APP_HOME%\lib\jfontchooser-1.0.5.jar;%APP_HOME%\lib\jadx-cli-1.3.5.jar;%APP_HOME%\lib\jadx-core-1.3.5.jar;%APP_HOME%\lib\logback-classic-1.2.11.jar;%APP_HOME%\lib\jadx-java-convert-1.3.5.jar;%APP_HOME%\lib\jadx-smali-input-1.3.5.jar;%APP_HOME%\lib\jadx-dex-input-1.3.5.jar;%APP_HOME%\lib\jadx-java-input-1.3.5.jar;%APP_HOME%\lib\jadx-plugins-api-1.3.5.jar;%APP_HOME%\lib\raung-disasm-0.0.2.jar;%APP_HOME%\lib\raung-common-0.0.2.jar;%APP_HOME%\lib\slf4j-api-1.7.36.jar;%APP_HOME%\lib\baksmali-2.5.2.jar;%APP_HOME%\lib\smali-2.5.2.jar;%APP_HOME%\lib\util-2.5.2.jar;%APP_HOME%\lib\jcommander-1.82.jar;%APP_HOME%\lib\rsyntaxtextarea-3.2.0.jar;%APP_HOME%\lib\image-viewer-1.2.3.jar;%APP_HOME%\lib\flatlaf-intellij-themes-2.1.jar;%APP_HOME%\lib\flatlaf-extras-2.1.jar;%APP_HOME%\lib\flatlaf-2.1.jar;%APP_HOME%\lib\svgSalamander-1.1.3.jar;%APP_HOME%\lib\gson-2.9.0.jar;%APP_HOME%\lib\commons-text-1.9.jar;%APP_HOME%\lib\commons-lang3-3.12.0.jar;%APP_HOME%\lib\rxjava2-swing-0.3.7.jar;%APP_HOME%\lib\rxjava-2.2.21.jar;%APP_HOME%\lib\apksig-4.2.1.jar;%APP_HOME%\lib\jdwp-1.0.jar;%APP_HOME%\lib\aapt2-proto-4.2.1-7147631.jar;%APP_HOME%\lib\protobuf-java-3.11.4.jar;%APP_HOME%\lib\logback-core-1.2.11.jar;%APP_HOME%\lib\reactive-streams-1.0.3.jar;%APP_HOME%\lib\dexlib2-2.5.2.jar;%APP_HOME%\lib\guava-30.1.1-jre.jar;%APP_HOME%\lib\dalvik-dx-11.0.0_r3.jar;%APP_HOME%\lib\r8-3.3.28.jar;%APP_HOME%\lib\asm-9.3.jar;%APP_HOME%\lib\antlr-3.5.2.jar;%APP_HOME%\lib\ST4-4.0.8.jar;%APP_HOME%\lib\antlr-runtime-3.5.2.jar;%APP_HOME%\lib\stringtemplate-3.2.1.jar;%APP_HOME%\lib\jsr305-3.0.2.jar;%APP_HOME%\lib\failureaccess-1.0.1.jar;%APP_HOME%\lib\listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar;%APP_HOME%\lib\checker-qual-3.8.0.jar;%APP_HOME%\lib\error_prone_annotations-2.5.1.jar;%APP_HOME%\lib\j2objc-annotations-1.3.jar;%APP_HOME%\lib\antlr-2.7.7.jar + + +@rem Execute jadx-gui +start "jadx-gui" /B "%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %JADX_GUI_OPTS% -classpath "%CLASSPATH%" jadx.gui.JadxGUI %* + +:end +@rem End local scope for the variables with windows NT shell +if "%ERRORLEVEL%"=="0" goto mainEnd + +:fail +rem Set variable JADX_GUI_EXIT_CONSOLE if you need the _script_ return code instead of +rem the _cmd.exe /c_ return code! +if not "" == "%JADX_GUI_EXIT_CONSOLE%" exit 1 +exit /b 1 + +:mainEnd +if "%OS%"=="Windows_NT" endlocal + +:omega diff --git a/config/tools/jadx/bin/jadx.bat b/config/tools/jadx/bin/jadx.bat new file mode 100755 index 0000000..4cfe522 --- /dev/null +++ b/config/tools/jadx/bin/jadx.bat @@ -0,0 +1,89 @@ +@rem +@rem Copyright 2015 the original author or authors. +@rem +@rem Licensed under the Apache License, Version 2.0 (the "License"); +@rem you may not use this file except in compliance with the License. +@rem You may obtain a copy of the License at +@rem +@rem https://www.apache.org/licenses/LICENSE-2.0 +@rem +@rem Unless required by applicable law or agreed to in writing, software +@rem distributed under the License is distributed on an "AS IS" BASIS, +@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +@rem See the License for the specific language governing permissions and +@rem limitations under the License. +@rem + +@if "%DEBUG%" == "" @echo off +@rem ########################################################################## +@rem +@rem jadx startup script for Windows +@rem +@rem ########################################################################## + +@rem Set local scope for the variables with windows NT shell +if "%OS%"=="Windows_NT" setlocal + +set DIRNAME=%~dp0 +if "%DIRNAME%" == "" set DIRNAME=. +set APP_BASE_NAME=%~n0 +set APP_HOME=%DIRNAME%.. + +@rem Resolve any "." and ".." in APP_HOME to make it shorter. +for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi + +@rem Add default JVM options here. You can also use JAVA_OPTS and JADX_OPTS to pass JVM options to this script. +set DEFAULT_JVM_OPTS="-Xms128M" "-XX:MaxRAMPercentage=70.0" "-XX:+UseG1GC" + +@rem Find java.exe +if defined JAVA_HOME goto findJavaFromJavaHome + +set JAVA_EXE=java.exe +%JAVA_EXE% -version >NUL 2>&1 +if "%ERRORLEVEL%" == "0" goto execute + +echo. +echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. +echo. +echo Please set the JAVA_HOME variable in your environment to match the +echo location of your Java installation. + +goto fail + +:findJavaFromJavaHome +set JAVA_HOME=%JAVA_HOME:"=% +set JAVA_EXE=%JAVA_HOME%/bin/java.exe + +if exist "%JAVA_EXE%" goto execute + +echo. +echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% +echo. +echo Please set the JAVA_HOME variable in your environment to match the +echo location of your Java installation. + +goto fail + +:execute +@rem Setup the command line + +set CLASSPATH=%APP_HOME%\lib\jadx-cli-1.3.5.jar;%APP_HOME%\lib\jadx-core-1.3.5.jar;%APP_HOME%\lib\logback-classic-1.2.11.jar;%APP_HOME%\lib\jadx-java-convert-1.3.5.jar;%APP_HOME%\lib\jadx-smali-input-1.3.5.jar;%APP_HOME%\lib\jadx-dex-input-1.3.5.jar;%APP_HOME%\lib\jadx-java-input-1.3.5.jar;%APP_HOME%\lib\jadx-plugins-api-1.3.5.jar;%APP_HOME%\lib\raung-disasm-0.0.2.jar;%APP_HOME%\lib\raung-common-0.0.2.jar;%APP_HOME%\lib\slf4j-api-1.7.36.jar;%APP_HOME%\lib\baksmali-2.5.2.jar;%APP_HOME%\lib\smali-2.5.2.jar;%APP_HOME%\lib\util-2.5.2.jar;%APP_HOME%\lib\jcommander-1.82.jar;%APP_HOME%\lib\gson-2.9.0.jar;%APP_HOME%\lib\aapt2-proto-4.2.1-7147631.jar;%APP_HOME%\lib\protobuf-java-3.11.4.jar;%APP_HOME%\lib\logback-core-1.2.11.jar;%APP_HOME%\lib\dexlib2-2.5.2.jar;%APP_HOME%\lib\guava-30.1.1-jre.jar;%APP_HOME%\lib\dalvik-dx-11.0.0_r3.jar;%APP_HOME%\lib\r8-3.3.28.jar;%APP_HOME%\lib\asm-9.3.jar;%APP_HOME%\lib\antlr-3.5.2.jar;%APP_HOME%\lib\ST4-4.0.8.jar;%APP_HOME%\lib\antlr-runtime-3.5.2.jar;%APP_HOME%\lib\stringtemplate-3.2.1.jar;%APP_HOME%\lib\jsr305-3.0.2.jar;%APP_HOME%\lib\failureaccess-1.0.1.jar;%APP_HOME%\lib\listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar;%APP_HOME%\lib\checker-qual-3.8.0.jar;%APP_HOME%\lib\error_prone_annotations-2.5.1.jar;%APP_HOME%\lib\j2objc-annotations-1.3.jar;%APP_HOME%\lib\antlr-2.7.7.jar + + +@rem Execute jadx +"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %JADX_OPTS% -classpath "%CLASSPATH%" jadx.cli.JadxCLI %* + +:end +@rem End local scope for the variables with windows NT shell +if "%ERRORLEVEL%"=="0" goto mainEnd + +:fail +rem Set variable JADX_EXIT_CONSOLE if you need the _script_ return code instead of +rem the _cmd.exe /c_ return code! +if not "" == "%JADX_EXIT_CONSOLE%" exit 1 +exit /b 1 + +:mainEnd +if "%OS%"=="Windows_NT" endlocal + +:omega diff --git a/config/tools/jadx/bin/jadx.sh b/config/tools/jadx/bin/jadx.sh new file mode 100755 index 0000000..86a9fcf --- /dev/null +++ b/config/tools/jadx/bin/jadx.sh @@ -0,0 +1,35 @@ +#!/usr/bin/env sh + +jadx=$1 +apk=$2 +output=$3 +ds=$4 + +done_file=$output/.done + +if [ -f "$done_file" ] +then + exit 0 +fi + +rm -rf "$output" + +#timeout -s SIGKILL 1800 $jadx --quiet \ +# --no-imports \ +# --no-res \ +# --show-bad-code \ +# --no-debug-info \ +# --output-dir-src "$output" \ +# --threads-count "$ds" \ +# "$apk" + +timeout -s SIGKILL 1800 $jadx --quiet \ + --no-imports \ + --show-bad-code \ + --no-debug-info \ + --output-dir "$output" \ + --threads-count "$ds" \ + --export-gradle \ + "$apk" + +touch "$done_file" diff --git a/config/tools/jadx/lib/ST4-4.0.8.jar b/config/tools/jadx/lib/ST4-4.0.8.jar new file mode 100644 index 0000000..144828b Binary files /dev/null and b/config/tools/jadx/lib/ST4-4.0.8.jar differ diff --git a/config/tools/jadx/lib/aapt2-proto-4.2.1-7147631.jar b/config/tools/jadx/lib/aapt2-proto-4.2.1-7147631.jar new file mode 100644 index 0000000..acfd5e7 Binary files /dev/null and b/config/tools/jadx/lib/aapt2-proto-4.2.1-7147631.jar differ diff --git a/config/tools/jadx/lib/antlr-2.7.7.jar b/config/tools/jadx/lib/antlr-2.7.7.jar new file mode 100644 index 0000000..5e5f14b Binary files /dev/null and b/config/tools/jadx/lib/antlr-2.7.7.jar differ diff --git a/config/tools/jadx/lib/antlr-3.5.2.jar b/config/tools/jadx/lib/antlr-3.5.2.jar new file mode 100644 index 0000000..12b6140 Binary files /dev/null and b/config/tools/jadx/lib/antlr-3.5.2.jar differ diff --git a/config/tools/jadx/lib/antlr-runtime-3.5.2.jar b/config/tools/jadx/lib/antlr-runtime-3.5.2.jar new file mode 100644 index 0000000..d48e3e8 Binary files /dev/null and b/config/tools/jadx/lib/antlr-runtime-3.5.2.jar differ diff --git a/config/tools/jadx/lib/apksig-4.2.1.jar b/config/tools/jadx/lib/apksig-4.2.1.jar new file mode 100644 index 0000000..159148c Binary files /dev/null and b/config/tools/jadx/lib/apksig-4.2.1.jar differ diff --git a/config/tools/jadx/lib/asm-9.3.jar b/config/tools/jadx/lib/asm-9.3.jar new file mode 100644 index 0000000..bd8b948 Binary files /dev/null and b/config/tools/jadx/lib/asm-9.3.jar differ diff --git a/config/tools/jadx/lib/baksmali-2.5.2.jar b/config/tools/jadx/lib/baksmali-2.5.2.jar new file mode 100644 index 0000000..bd686f3 Binary files /dev/null and b/config/tools/jadx/lib/baksmali-2.5.2.jar differ diff --git a/config/tools/jadx/lib/checker-qual-3.8.0.jar b/config/tools/jadx/lib/checker-qual-3.8.0.jar new file mode 100644 index 0000000..d30059e Binary files /dev/null and b/config/tools/jadx/lib/checker-qual-3.8.0.jar differ diff --git a/config/tools/jadx/lib/commons-lang3-3.12.0.jar b/config/tools/jadx/lib/commons-lang3-3.12.0.jar new file mode 100644 index 0000000..4d434a2 Binary files /dev/null and b/config/tools/jadx/lib/commons-lang3-3.12.0.jar differ diff --git a/config/tools/jadx/lib/commons-text-1.9.jar b/config/tools/jadx/lib/commons-text-1.9.jar new file mode 100644 index 0000000..cc0c690 Binary files /dev/null and b/config/tools/jadx/lib/commons-text-1.9.jar differ diff --git a/config/tools/jadx/lib/dalvik-dx-11.0.0_r3.jar b/config/tools/jadx/lib/dalvik-dx-11.0.0_r3.jar new file mode 100644 index 0000000..90d9317 Binary files /dev/null and b/config/tools/jadx/lib/dalvik-dx-11.0.0_r3.jar differ diff --git a/config/tools/jadx/lib/dexlib2-2.5.2.jar b/config/tools/jadx/lib/dexlib2-2.5.2.jar new file mode 100644 index 0000000..392b3fc Binary files /dev/null and b/config/tools/jadx/lib/dexlib2-2.5.2.jar differ diff --git a/config/tools/jadx/lib/error_prone_annotations-2.5.1.jar b/config/tools/jadx/lib/error_prone_annotations-2.5.1.jar new file mode 100644 index 0000000..fbc220c Binary files /dev/null and b/config/tools/jadx/lib/error_prone_annotations-2.5.1.jar differ diff --git a/config/tools/jadx/lib/failureaccess-1.0.1.jar b/config/tools/jadx/lib/failureaccess-1.0.1.jar new file mode 100644 index 0000000..9b56dc7 Binary files /dev/null and b/config/tools/jadx/lib/failureaccess-1.0.1.jar differ diff --git a/config/tools/jadx/lib/flatlaf-2.1.jar b/config/tools/jadx/lib/flatlaf-2.1.jar new file mode 100644 index 0000000..bf7da70 Binary files /dev/null and b/config/tools/jadx/lib/flatlaf-2.1.jar differ diff --git a/config/tools/jadx/lib/flatlaf-extras-2.1.jar b/config/tools/jadx/lib/flatlaf-extras-2.1.jar new file mode 100644 index 0000000..904428d Binary files /dev/null and b/config/tools/jadx/lib/flatlaf-extras-2.1.jar differ diff --git a/config/tools/jadx/lib/flatlaf-intellij-themes-2.1.jar b/config/tools/jadx/lib/flatlaf-intellij-themes-2.1.jar new file mode 100644 index 0000000..28caa51 Binary files /dev/null and b/config/tools/jadx/lib/flatlaf-intellij-themes-2.1.jar differ diff --git a/config/tools/jadx/lib/gson-2.9.0.jar b/config/tools/jadx/lib/gson-2.9.0.jar new file mode 100644 index 0000000..fb62e05 Binary files /dev/null and b/config/tools/jadx/lib/gson-2.9.0.jar differ diff --git a/config/tools/jadx/lib/guava-30.1.1-jre.jar b/config/tools/jadx/lib/guava-30.1.1-jre.jar new file mode 100644 index 0000000..93ebf3b Binary files /dev/null and b/config/tools/jadx/lib/guava-30.1.1-jre.jar differ diff --git a/config/tools/jadx/lib/image-viewer-1.2.3.jar b/config/tools/jadx/lib/image-viewer-1.2.3.jar new file mode 100644 index 0000000..073f756 Binary files /dev/null and b/config/tools/jadx/lib/image-viewer-1.2.3.jar differ diff --git a/config/tools/jadx/lib/j2objc-annotations-1.3.jar b/config/tools/jadx/lib/j2objc-annotations-1.3.jar new file mode 100644 index 0000000..a429c72 Binary files /dev/null and b/config/tools/jadx/lib/j2objc-annotations-1.3.jar differ diff --git a/config/tools/jadx/lib/jadx-cli-1.3.5.jar b/config/tools/jadx/lib/jadx-cli-1.3.5.jar new file mode 100644 index 0000000..ba1e320 Binary files /dev/null and b/config/tools/jadx/lib/jadx-cli-1.3.5.jar differ diff --git a/config/tools/jadx/lib/jadx-core-1.3.5.jar b/config/tools/jadx/lib/jadx-core-1.3.5.jar new file mode 100644 index 0000000..e61cbf8 Binary files /dev/null and b/config/tools/jadx/lib/jadx-core-1.3.5.jar differ diff --git a/config/tools/jadx/lib/jadx-dex-input-1.3.5.jar b/config/tools/jadx/lib/jadx-dex-input-1.3.5.jar new file mode 100644 index 0000000..1561620 Binary files /dev/null and b/config/tools/jadx/lib/jadx-dex-input-1.3.5.jar differ diff --git a/config/tools/jadx/lib/jadx-gui-1.3.5.jar b/config/tools/jadx/lib/jadx-gui-1.3.5.jar new file mode 100644 index 0000000..c2ad1e8 Binary files /dev/null and b/config/tools/jadx/lib/jadx-gui-1.3.5.jar differ diff --git a/config/tools/jadx/lib/jadx-java-convert-1.3.5.jar b/config/tools/jadx/lib/jadx-java-convert-1.3.5.jar new file mode 100644 index 0000000..ab06fc2 Binary files /dev/null and b/config/tools/jadx/lib/jadx-java-convert-1.3.5.jar differ diff --git a/config/tools/jadx/lib/jadx-java-input-1.3.5.jar b/config/tools/jadx/lib/jadx-java-input-1.3.5.jar new file mode 100644 index 0000000..fe1406f Binary files /dev/null and b/config/tools/jadx/lib/jadx-java-input-1.3.5.jar differ diff --git a/config/tools/jadx/lib/jadx-plugins-api-1.3.5.jar b/config/tools/jadx/lib/jadx-plugins-api-1.3.5.jar new file mode 100644 index 0000000..6706f51 Binary files /dev/null and b/config/tools/jadx/lib/jadx-plugins-api-1.3.5.jar differ diff --git a/config/tools/jadx/lib/jadx-smali-input-1.3.5.jar b/config/tools/jadx/lib/jadx-smali-input-1.3.5.jar new file mode 100644 index 0000000..862917b Binary files /dev/null and b/config/tools/jadx/lib/jadx-smali-input-1.3.5.jar differ diff --git a/config/tools/jadx/lib/jcommander-1.82.jar b/config/tools/jadx/lib/jcommander-1.82.jar new file mode 100644 index 0000000..90f4424 Binary files /dev/null and b/config/tools/jadx/lib/jcommander-1.82.jar differ diff --git a/config/tools/jadx/lib/jdwp-1.0.jar b/config/tools/jadx/lib/jdwp-1.0.jar new file mode 100644 index 0000000..60917a8 Binary files /dev/null and b/config/tools/jadx/lib/jdwp-1.0.jar differ diff --git a/config/tools/jadx/lib/jfontchooser-1.0.5.jar b/config/tools/jadx/lib/jfontchooser-1.0.5.jar new file mode 100644 index 0000000..3a9779f Binary files /dev/null and b/config/tools/jadx/lib/jfontchooser-1.0.5.jar differ diff --git a/config/tools/jadx/lib/jsr305-3.0.2.jar b/config/tools/jadx/lib/jsr305-3.0.2.jar new file mode 100644 index 0000000..59222d9 Binary files /dev/null and b/config/tools/jadx/lib/jsr305-3.0.2.jar differ diff --git a/config/tools/jadx/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar b/config/tools/jadx/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar new file mode 100644 index 0000000..45832c0 Binary files /dev/null and b/config/tools/jadx/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar differ diff --git a/config/tools/jadx/lib/logback-classic-1.2.11.jar b/config/tools/jadx/lib/logback-classic-1.2.11.jar new file mode 100644 index 0000000..b70c0e6 Binary files /dev/null and b/config/tools/jadx/lib/logback-classic-1.2.11.jar differ diff --git a/config/tools/jadx/lib/logback-core-1.2.11.jar b/config/tools/jadx/lib/logback-core-1.2.11.jar new file mode 100644 index 0000000..e3038da Binary files /dev/null and b/config/tools/jadx/lib/logback-core-1.2.11.jar differ diff --git a/config/tools/jadx/lib/protobuf-java-3.11.4.jar b/config/tools/jadx/lib/protobuf-java-3.11.4.jar new file mode 100644 index 0000000..7224d23 Binary files /dev/null and b/config/tools/jadx/lib/protobuf-java-3.11.4.jar differ diff --git a/config/tools/jadx/lib/r8-3.3.28.jar b/config/tools/jadx/lib/r8-3.3.28.jar new file mode 100644 index 0000000..e4535bc Binary files /dev/null and b/config/tools/jadx/lib/r8-3.3.28.jar differ diff --git a/config/tools/jadx/lib/raung-common-0.0.2.jar b/config/tools/jadx/lib/raung-common-0.0.2.jar new file mode 100644 index 0000000..58a22c9 Binary files /dev/null and b/config/tools/jadx/lib/raung-common-0.0.2.jar differ diff --git a/config/tools/jadx/lib/raung-disasm-0.0.2.jar b/config/tools/jadx/lib/raung-disasm-0.0.2.jar new file mode 100644 index 0000000..e7d8da6 Binary files /dev/null and b/config/tools/jadx/lib/raung-disasm-0.0.2.jar differ diff --git a/config/tools/jadx/lib/reactive-streams-1.0.3.jar b/config/tools/jadx/lib/reactive-streams-1.0.3.jar new file mode 100644 index 0000000..b9b487c Binary files /dev/null and b/config/tools/jadx/lib/reactive-streams-1.0.3.jar differ diff --git a/config/tools/jadx/lib/rsyntaxtextarea-3.2.0.jar b/config/tools/jadx/lib/rsyntaxtextarea-3.2.0.jar new file mode 100644 index 0000000..fce14e0 Binary files /dev/null and b/config/tools/jadx/lib/rsyntaxtextarea-3.2.0.jar differ diff --git a/config/tools/jadx/lib/rxjava-2.2.21.jar b/config/tools/jadx/lib/rxjava-2.2.21.jar new file mode 100644 index 0000000..377d311 Binary files /dev/null and b/config/tools/jadx/lib/rxjava-2.2.21.jar differ diff --git a/config/tools/jadx/lib/rxjava2-swing-0.3.7.jar b/config/tools/jadx/lib/rxjava2-swing-0.3.7.jar new file mode 100644 index 0000000..db43fa6 Binary files /dev/null and b/config/tools/jadx/lib/rxjava2-swing-0.3.7.jar differ diff --git a/config/tools/jadx/lib/slf4j-api-1.7.36.jar b/config/tools/jadx/lib/slf4j-api-1.7.36.jar new file mode 100644 index 0000000..7d3ce68 Binary files /dev/null and b/config/tools/jadx/lib/slf4j-api-1.7.36.jar differ diff --git a/config/tools/jadx/lib/smali-2.5.2.jar b/config/tools/jadx/lib/smali-2.5.2.jar new file mode 100644 index 0000000..ec0da93 Binary files /dev/null and b/config/tools/jadx/lib/smali-2.5.2.jar differ diff --git a/config/tools/jadx/lib/stringtemplate-3.2.1.jar b/config/tools/jadx/lib/stringtemplate-3.2.1.jar new file mode 100644 index 0000000..d0e11b7 Binary files /dev/null and b/config/tools/jadx/lib/stringtemplate-3.2.1.jar differ diff --git a/config/tools/jadx/lib/svgSalamander-1.1.3.jar b/config/tools/jadx/lib/svgSalamander-1.1.3.jar new file mode 100644 index 0000000..3a4d29a Binary files /dev/null and b/config/tools/jadx/lib/svgSalamander-1.1.3.jar differ diff --git a/config/tools/jadx/lib/util-2.5.2.jar b/config/tools/jadx/lib/util-2.5.2.jar new file mode 100644 index 0000000..51e993e Binary files /dev/null and b/config/tools/jadx/lib/util-2.5.2.jar differ diff --git a/config/tools/platforms/android-14/android.jar b/config/tools/platforms/android-14/android.jar new file mode 100644 index 0000000..1995619 Binary files /dev/null and b/config/tools/platforms/android-14/android.jar differ diff --git a/config/tools/platforms/android-16/android.jar b/config/tools/platforms/android-16/android.jar new file mode 100644 index 0000000..7b08d15 Binary files /dev/null and b/config/tools/platforms/android-16/android.jar differ diff --git a/config/tools/platforms/android-17/android.jar b/config/tools/platforms/android-17/android.jar new file mode 100644 index 0000000..bc15cc5 Binary files /dev/null and b/config/tools/platforms/android-17/android.jar differ diff --git a/config/tools/platforms/android-18/android.jar b/config/tools/platforms/android-18/android.jar new file mode 100644 index 0000000..29bb4c6 Binary files /dev/null and b/config/tools/platforms/android-18/android.jar differ diff --git a/config/tools/platforms/android-19/android.jar b/config/tools/platforms/android-19/android.jar new file mode 100644 index 0000000..ca5ceca Binary files /dev/null and b/config/tools/platforms/android-19/android.jar differ diff --git a/config/tools/platforms/android-20/android.jar b/config/tools/platforms/android-20/android.jar new file mode 100644 index 0000000..7cbbb64 Binary files /dev/null and b/config/tools/platforms/android-20/android.jar differ diff --git a/config/tools/platforms/android-22/android.jar b/config/tools/platforms/android-22/android.jar new file mode 100644 index 0000000..1dc21fd Binary files /dev/null and b/config/tools/platforms/android-22/android.jar differ diff --git a/config/tools/platforms/android-23/android.jar b/config/tools/platforms/android-23/android.jar new file mode 100644 index 0000000..3e2434d Binary files /dev/null and b/config/tools/platforms/android-23/android.jar differ diff --git a/config/tools/platforms/android-24/android.jar b/config/tools/platforms/android-24/android.jar new file mode 100644 index 0000000..05855c9 Binary files /dev/null and b/config/tools/platforms/android-24/android.jar differ diff --git a/config/tools/platforms/android-25/android.jar b/config/tools/platforms/android-25/android.jar new file mode 100644 index 0000000..a35512c Binary files /dev/null and b/config/tools/platforms/android-25/android.jar differ diff --git a/config/tools/platforms/android-26/android.jar b/config/tools/platforms/android-26/android.jar new file mode 100644 index 0000000..a0c3df3 Binary files /dev/null and b/config/tools/platforms/android-26/android.jar differ diff --git a/config/tools/platforms/android-27/android.jar b/config/tools/platforms/android-27/android.jar new file mode 100644 index 0000000..bd1d4ee Binary files /dev/null and b/config/tools/platforms/android-27/android.jar differ diff --git a/config/tools/platforms/android-28/android.jar b/config/tools/platforms/android-28/android.jar new file mode 100644 index 0000000..66e18e0 Binary files /dev/null and b/config/tools/platforms/android-28/android.jar differ diff --git a/config/tools/platforms/android-29/android.jar b/config/tools/platforms/android-29/android.jar new file mode 100644 index 0000000..345dd11 Binary files /dev/null and b/config/tools/platforms/android-29/android.jar differ diff --git a/config/tools/platforms/android-30/android.jar b/config/tools/platforms/android-30/android.jar new file mode 100644 index 0000000..5e69644 Binary files /dev/null and b/config/tools/platforms/android-30/android.jar differ diff --git a/config/tools/platforms/android-31/android.jar b/config/tools/platforms/android-31/android.jar new file mode 100644 index 0000000..af4ff4f Binary files /dev/null and b/config/tools/platforms/android-31/android.jar differ diff --git a/config/tools/platforms/android-32/android.jar b/config/tools/platforms/android-32/android.jar new file mode 100644 index 0000000..34f5d75 Binary files /dev/null and b/config/tools/platforms/android-32/android.jar differ diff --git a/config/tools/platforms/android-33/android.jar b/config/tools/platforms/android-33/android.jar new file mode 100644 index 0000000..dd15001 Binary files /dev/null and b/config/tools/platforms/android-33/android.jar differ diff --git a/doc/en/EngineConfig.md b/doc/en/EngineConfig.md new file mode 100644 index 0000000..651e3c1 --- /dev/null +++ b/doc/en/EngineConfig.md @@ -0,0 +1,136 @@ +# EngineConfig Description +EngineConfig.json5 is a more in-depth configuration of the engine. If you do not understand the exact meaning, it is recommended to keep this file unchanged. + +## IgnoreList +The methods that match the keywords will be ignored during the analysis. The ignoreLise can be specified using keywords with different granularities. +- PackageName: all methods in this package will be ignored +- MethodName: all methods with this name will be neglected +- MethodSignature: methods with this signature will not be considered + +## Callback +When an implemented class of an interface is created and called implicitly, insert an explicit call statement automatically. For example, +```json + "java.lang.Runnable": [ + "void run()" + ] +``` +```java +Runnable r=new Runnable(){ + @Override + public void run() { + //do something + } +}; +// insert the statement below to call explicitly +// r.run(); +``` + +## Library + +Specify packages to be considered as library. This is to reduce the analysis workload of Appshark, because it is believed that code in framework or jdk do not contain vulnerabilities. +If a package is marked as library, it will have the following impacts: +1. Ignore library code when constructing the call graph +2. Sources and sinks can be matched for library only +3. When encountering library method calls, we will apply the propagation rules (i.e. PointerFlowRule and VariableFlowRule), instead of their implementation, to calculate pointer and data flow info between variables. + +## PointerFlowRule +**Modify the rule only when you have a strong reason** +The rule describes how to handle the pointer information in a method call when the callee method is from a library or exceeds the `traceDepth` from the entry. +Basic principle: **if there is no matching rule, then the related points-to relationship is not considered** + +Keywords in this kind of rule: + - p*: all parameters +- @this: this pointer in a method call +- ret: the return value of the method +- p0,p1,p2: 0th,1st,2nd argument of a method +- @this.data: a dummy field representing this object's all fields +- I: input +- O: output + +Like above, MethodName or MethodSig can be specified to match methods by a method name or a method signature. + +An example for keySet(): +```json + "": { + "@this.data->ret": { + "I": [ + "@this.data" + ], + "O": [ + "ret" + ] + } + } +``` + +When encountering the below code: +```java +HashMap m=new HashMap(); +m.put("s1","s2"); +m.put("i",new Intent()); +Set s= m.keySet(); +``` +Variable s (i.e. the return value of keySet()) points to m's all fields. If `@this.data` points to objects `("s1","s2","i",new Intent())`, then s points to these objects as well. + + +## VariableFlowRule + +The rule describes how to handle the data flow information in a method call when the callee method is from a library or exceeds the `traceDepth` from the entry. +The difference between it and PointerFlowRule is that it has default data flow relationship, including: +- InstantDefault +- InstantSelfDefault + +The difference between InstantSelfDefault and InstantDefault is that the former explicitly requires the `this` pointers of caller and callee to be the same. Say, +```java +class C{ + void f(){ + g(); + } + void g(){ + + } +} +``` +Since f() and g() are from the same instance, the data flow relationship is InstantSelfDefault rather than InstantDefault. + +**Same as PointerFlowRule, keywords in this kind of rule:** + + - p*: all parameters +- @this: this pointer in a method call +- ret: the return value of the method +- p0,p1,p2: 0th,1st,2nd argument of a method +- @this.data: a dummy field representing this object's all fields +- I: input +- O: output + +Here is an example. +```json +"": { +"@this.data->ret": { + "I": [ + "@this.data" + ], + "O": [ + "ret" + ] +}, +"@this->ret": { + "I": [ + "@this" + ], + "O": [ + "ret" + ] +} +} +``` + +```java +HashMap m=new HashMap(); +m.put("s1","s2"); +m.put("i",new Intent()); +Object obj= m.remove("i3"); +``` +If `@this.data` points to objects `("s1","s2","i",new Intent())`, then data in the HashMap will flow from `("s1","s2","i",new Intent())` to obj, and then flow from m to obj. + +In other words, if `new HashMap()` is a tainted source, the obj will be tainted. If "s1" is a tainted source, then obj will also be tainted. diff --git a/doc/en/appshark_flow_chart.png b/doc/en/appshark_flow_chart.png new file mode 100644 index 0000000..3b79ec8 Binary files /dev/null and b/doc/en/appshark_flow_chart.png differ diff --git a/doc/en/argument.md b/doc/en/argument.md new file mode 100644 index 0000000..f3383fd --- /dev/null +++ b/doc/en/argument.md @@ -0,0 +1,161 @@ +# Running parameter description + +The configuration set by parameters and the EngineConfig file are adjustment of Appshark's parameters. Specifically, parameters are one time for the current app, while the EngineConfig is for all the apps. + +## Parameters required to be specified + +apkPath: the absolute path of the app to be scanned +rules: the rules applied to the scanning, which are a comma-separated list of rules + +## Other default parameters + +### CallBackEnhance + +If it is `true`, then it will insert a rewritten version of invocation statements to related methods in anonymous classes. In this way, we can model the implicit method calls. For example, + + ``` + sensitiveApi.setOnClickListener(new MyListener() { + public void onClick(View v) { + //call + } + }); + // will insert a line of an invocation below + // l.onClick(null); //l is the newly created instance of MyListener + ``` + + ### maxPointerAnalyzeTime + +It is the maximum timeout for pointer analysis, in seconds. If it is too short, it may lead to FPs and FNs because some of the code is not analyzed. + +### javaSource + +Whether to display the source code in the final vulnerability details. The source code is decompiled from [jadx](https://github.com/skylot/jadx) tool + +### maxThread + +It controls the degree of parallelism when performing analyses such as pointer analysis. Its default value is CPU core number of the running machine. If OOM happens, then we can lower its value to save memory. But the runtime will be longer. + +### rulePath + +The folder path of the rule files: it is given in an absolute path. Its default value is `lib/rules` in the project directory. + +### sdkPath + +The path of Android SDK: its default value is `tools/platforms` in the project directory. + +### toolsPath + +The folder path of the 3rd party tools: its default value is `tools` in the project directory. + +### supportFragment + +Whether to consider lifecycle callbacks in a Fragment class. It is similar to process lifecycle callbacks (e.g. onCreate) in an Activity class. + +### logLevel + +Log's output level, default is info (1): + +- debug 0 +- info 1 +- warn 2 +- error 3 + +### ruleMaxAnalyzer + +To prevent errors in a rule: if a rule is problematic, it may match too many sources and sinks. Its default value is 5,000, which should be good for most rules. +If you see `rule xxxx has too many rules: xxxx, dropped`, then you can consider increasing its value. + +### maxPathLength + +The longest path length from source to sink. Paths exceeding this length will be discarded even if they are valid. + +### wholeProcessMode + +whether to enable whole program analysis: false by default. The whole program analysis impacts the analysis scope. + +- May increase the analysis time considerably. It makes Appshark try to analyze each statement in the app, regardless of the relationship between source and sink specified by the user. +- More FPs and TPs due to the increased scope +- The pointer analysis step will be strictly single-threaded and cannot be parallelized + +#### Scenarios recommended to enable + +When the app size is small and the results can be obtained within your acceptable time and memory usage + +#### Scenarios recommended to disable + +When the app size is large and complex. Because it is almost impossible to get the results in an acceptable time. + +#### Comparison with SliceMode, DirectMode + +In a rule, no matter whether directly or indirectly, there are an analysis entry and an analysis trace depth specified. However, these two settings are ignored in wholeProcessMode +Clearly, the path finding step still uses source, sink, and sanitizer information in a rule. + +#### skipAnalyzeNonRelatedMethods + +There are so many methods that can be reached from the entry method within the path length of `traceDepth`. Is this necessary? An option is to only consider those reached methods that are related to sources and sinks. +Its default value is `false`, meaning taking all those methods into account. `True` indicates that skipping reached methods that have nothing to do with source and sink. Say, + +``` +Class C{ + void f(){ + Object o=source(); + g(o); + h(); + } + void g(Object o){ + sink(o); + } + void h(){ + + } +} +``` + +f() is the analysis entry. If skipAnalyzeNonRelatedMethods is true, h() will be neglected. Otherwise, h() will be analyzed. + +#### skipPointerPropagationForLibraryMethod +Currently, our pointer analysis is context insensitive. For library methods, such as `java.util.Iterator: java.lang.Object next()`, we can ignore the pointer information propagation in some cases to save some time. The potential side effect is to have more FPs and FNs. + +If the value is `true`, that is, do not propagate pointer information for library methods. We only track the pointer info defined by our predefined heuristics in the statement containing a library method call. + +```java +Class C{ + void f(){ + StringBuffer s1=new StringBuffer(); + StringBuffer s2=null; + StringBuffer s3=s2; + s2=s1.append("aa"); + String s=s3.toString(); + } +} +``` + +Note that EngineConfig.json5 includes a rule below: + +```json5 +{ + PointerFlowRule: { + "MethodSignature": { + "": { + "@this->ret": { + "I": [ + "@this" + ], + "O": [ + "ret" + ] + } + }, + }, + } +} +``` +When skipPointerPropagationForLibraryMethod is enabled, append() is a library method,, so we propagate as follows: +s1->{new StringBuffer();} // per StringBuffer s1=new StringBuffer(); +s2->{new StringBuffer();,null} // per s2=s1.append("aa"); and PointerFlowRule +s3->{null} + +When skipPointerPropagationForLibraryMethod is off, then we propagate as follows: +s1->{new StringBuffer();} // per StringBuffer s1=new StringBuffer(); +s2->{new StringBuffer();,null} // per s2=s1.append("aa"); and PointerFlowRule +s3->{new StringBuffer();,null} \ No newline at end of file diff --git a/doc/en/faq.md b/doc/en/faq.md new file mode 100644 index 0000000..0f9ac60 --- /dev/null +++ b/doc/en/faq.md @@ -0,0 +1,45 @@ +# FAQ + +## 1. How does Appshark differ from Soot? +Appshark analyzes the Jimple IR generated by Soot. Appshark relies on Soot, but Appshark has its independent rule system as well as independently implemented pointer analysis and taint analysis. + +## 2. How does Appshark differ from Flowdroid? +Appshark reuses Flowdroid's parsing module to process the AndroidManifest.xml file and various resource files. However, Appshark has its independent rule system as well as independently implemented pointer analysis and taint analysis. + +## 3. What problem might be caused by sliceMode? + +```java +public class SliceMode { + public static void main(String[] args) { + A x=new A(); + A y=x; + f(x,y); + } + void f(A x,A y){ + dosource(x); + dosink(y); + } + void dosource(A x){ + x.f=source(); + } + void dosink(A y){ + sink(y.f); + } +} +``` + +SliceMode treats f() as the entry method, then it fails to discover that x and y point to the same object. Thus, it cannot detect a feasible path from source and sink, leading to a FN. + +## 4. Field sensitivity +Appshark's analysis is field sensitive, and does not mix up the propagation of different fields. +For example, +```java +public class FieldSensitive{ + public static void main(){ + A x=new A(); + x.f1=source(); + sink(x.f2); + } +} +``` +Appshark will not detect a path from source to sink, because x.f1 and x.f2 are different fields. \ No newline at end of file diff --git a/doc/en/how_to_find_compliance_problem_use_appshark.md b/doc/en/how_to_find_compliance_problem_use_appshark.md new file mode 100644 index 0000000..ce93965 --- /dev/null +++ b/doc/en/how_to_find_compliance_problem_use_appshark.md @@ -0,0 +1,68 @@ +# How does Appshard identify the privacy compliance issues in an app + +Appshark can track data flows in an app, or identify the call site of an API call. Both features help you detect potential privacy compliance risks in your app. Privacy compliance-related rules are written in the same way as security vulnerability rules, so before you start, read [how_to_write_rules](). +## Privacy data flow analysis +Privacy data flow analysis is one of the data flow analyses. But most of the time you don't need to specify entry and sanitizer, you should care more about source and sink in rules. +Specifically, you can specify the source as an API obtaining private information. Say, +``` +"source": { + "Return": [ + "" + ] +} +``` +This API's return value is the unique IMEI number of a device. +Specify the method you think will leak private data as a sink, such as writing to a file: +``` +"sink": { + "": { + "TaintCheck": [ + "p0" + ] + } +} +``` +When the source of privacy data you are concerned about is not the API, but a field of an object, you can still write the rules according to the format of the field type source in the general rules, such as the device serial number as the source: +``` +"source": { + "Field": [ + "" + ] +} +``` +At last, you need to use sliceMode to reduce the analysis time. + + +Full rule file: +```json +{ + "getDeviceId": { + "SliceMode": true, + "traceDepth": 8, + "desc": { + "name": "getDeviceId", + "category": "ComplianceInfo", + "detail": "", + "wiki": "", + "complianceCategory": "PersonalDeviceInformation", + "complianceCategoryDetail": "PersonalDeviceInformation", + "level": "3" + }, + "source": { + "Return": [ + "" + ], + "Field": [ + "" + ] + }, + "sink": { + "": { + "TaintCheck": [ + "p0" + ] + } + } + } +} +``` diff --git a/doc/en/how_to_write_rules.md b/doc/en/how_to_write_rules.md new file mode 100644 index 0000000..488cae8 --- /dev/null +++ b/doc/en/how_to_write_rules.md @@ -0,0 +1,451 @@ +# How to write a rule for Appshark + +When using Appshark to scan apps, it is important to clearly tell Appshark the info you care about: the analysis entry methods, sources, sinks, and sanitizers. We divide sources into the following categories: + +- ConstStringMode: mark string literals as source +- ConstNumberMode: mark number literals as source +- SliceMode and DirectMode: mark other source types + +## Background + +The input to Appshark is Jimple statements in SSA form. Therefore, the signatures of methods and fields specified as source/sink must be in Jimple format. + +### Jimple method signature + +`` +This is a generic Java method signature, including a class name, a method name, a method return type, and a parameter type list. When specifying source and sink, each part can be specified as * to do fuzzy matching. +For example, `<*: android.content.Intent parseUri(java.lang.String,int)>` matches methods with a name `parseUri`, a return type `android.content.Intent`, and a parameter type list `java.lang.String,int` in any class. + +### Jimple field signature + +`` +This is a generic Java field signature of an object. Its owner class is `com.security.TestClass`. Its type is `android.content.Intent`. Its name is `fieldName`. +**Field signature does not support fuzzy matching** + +## Writing general rules + +The general rule consists of four parts, namely: 1) the entry of the analysis, 2) the source, 3) the sink, and 4) the sanitizer. + +### Specifying the analysis entry + +The entry of the analysis is generally a method. For example, + +```json +"entry": { +"methods": [ +"" +] +} +``` + +The entry only needs to be specified in the `DirectMode`. In the other three modes, there is no need to specify the analysis entry. If you don't know what the analysis entry is, you should not use the `DirectMode`. + +### General source specification +**It is worth noting that the real source in Appshark is specific variables, so no matter which way the source is written in, it will be converted into specific variables.** +The source can have many types, namely: + +- Constant string (note that this has nothing to do with `ConstStringMode`) +- The method return value +- A parameter of the function +- A field of an object +- Creation of an object + + +Five examples are described below. + +#### Constant string + +``json +"source": { +"ConstString": ["path1"] +} +``` + +In the code: + +```java +String s="path1"; +f(12,"path1"); +``` + +S is the source. +Parameter 1 of method f is also the source. + +#### Method return value + +This is one of the most common source forms, such as: + +```java +"source": { + "Return": [ + "" + ] +} +``` +It denotes that the return value of getName is the source. +In the code: +```java +ZipEntry e=getEntry(); +String name=e.getName(); +``` +The variable `name` is the source. + +#### A field of an object +For example, +```json + "source": { + "Field": [ + "", + ] + } +``` +In the code: +```java +Uri uri=CalendarContract.CONTENT_URI; +``` +Variable is the source. +**Note that it does not distinguish whether the field is static or non-static** + +#### Parameters of a method +Method parameters are generally used as the source in the case of rewriting system classes. +For example, +```json + "source": { + "Param": { + "": [ + "p1" + ] + } +} +``` +First of all, note that p0 is the first parameter, p1 is the second parameter. The parameter with the type of WebResourceRequest is the source. + +### Creation of an object +This rule is very special and is generally not used. +For example, +```json + "source": { + "NewInstance": ["android.content.Intent"] + } +``` +In the code, +```java +android.content.Intent i=new android.content.Intent(); +``` +Variable i is the source. + +### General sink specification +At present, the sink can only be the parameter(s) or the base object of a method, which can be: +- this pointer: @this +- some parameter of a method: p0,p1,p2 +- all parameters of a method: p* + +#### Sink +Note that **all sinks will be converted to specific variables internally**. +The specification of the sink is much simpler than that of the source, and the types are relatively simple. +For example, +```json + "sink": { + "(*)>": { + "LibraryOnly":true, + "TaintCheck": [ + "p*" + ] + }, + "(*)>": { + "TaintCheck": [ + "p*","@this" + ] + } + } +``` +In the code, + ```java +String path; +File f=new File(path); +FileOutputStream fileOutputStream=new FileOutputStream(f); + ``` +Variables f and fileOutputStream are sinks. Appshark checks whether it can find a taint propagation path from the source to these variables. + +Another configurable option for a sink is `LibraryOnly`. Its default value is false. If it is true, then the matching method signature must be the Library specified in the `EngineConfig.json5` file. +Taking the above as an example, if `com.security` is specified as Library in the `EngineConfig.json5` file, then the variable path is a sink. +Otherwise, the variable path is not a sink. + +### Sanitizer specification + +The purpose of sanitizer is to eliminate false positives. Although a complete propagation path from source to sink has been found, it can be invalid because the source has been strictly checked or filtered. +Let's take the unzipSlip rule as an example to introduce the principle of sanitizer. +The principle of the zip slip vulnerability can refer to [Directory traversal attack](https://en.wikipedia.org/wiki/Directory_traversal_attack). Mainly when decompressing a zip file, it does not check whether the file name contains "../", As a result, if the zip file is externally controllable, it may cause arbitrary file overwriting problems. + +First, give the complete rule: +```json +{ + "unZipSlip": { + "SliceMode": true, + "traceDepth": 8, + "desc": { + "name": "unZipSlip", + "category": "FileRisk", + "detail": "ZIP Slip is a highly critical security vulnerability aimed at these kinds of applications. ZIP Slip makes your application vulnerable to Path traversal attack and Sensitive data exposure.", + "wiki": "", + "possibility": "4", + "model": "middle" + }, + "source": { + "Return": [ + "" + ] + }, + "sanitize": { + "getCanonicalPath": { + "": { + "TaintCheck": [ + "@this" + ] + } + }, + "containsDotdot": { + "": { + "TaintCheck": [ + "@this" + ], + "p0": [ + "..*" + ] + } + }, + "indexDotdot": { + "": { + "TaintCheck": [ + "@this" + ], + "p0": [ + "..*" + ] + } + } + }, + "sink": { + "(*)>": { + "TaintCheck": [ + "p*" + ] + }, + "(*)>": { + "TaintCheck": [ + "p*", + "@this" + ] + } + } + } +} +``` + +We omit the discussion of sources and sinks, since we just introduced them above. We focus on explaining the sanitizer. + +#### The top-level rule is an OR relationship +The sanitizer contains three subkeys: +- getCanonicalPath +- containsDotdot +- indexDotdot +These three subkeys/rules work in an OR relationship. According to the rules, we may find N sources and M sinks. Then theoretically there will be N*M paths. For each of these paths, if it satisfies any of the three rules, then it will be sanitized. + +#### The relationship between the second-level rules is AND +Since there is only one second-level rule in this example, we create a rule for illustration. +```json + "containsDotdot": { + "": { + "TaintCheck": [ + "@this" + ], + "p0": [ + "..*" + ] + }, + "": { + "TaintCheck": [ + "@this" + ] + } + } +``` +If a path satisfies the two rules `` and ``, then this path will be santized. + +#### Meaning of specific rules +Again, **Appshark analyzes the taint propagation relationship between variables, so source, sink, and sanitizer are described at the variable level.** +Take containsDotdot as an example: +```json +"containsDotdot": { + "": { + "TaintCheck": [ + "@this" + ], + "p0": [ + "..*" + ] + } +} +``` +It has two requirements: +1. TaintCheck +Starting from the source, check whether the `this` pointer of the function `` is tainted among all the variables reached it. +For example, +```java +if(path.contains("../")){ + return false +} +``` +The path variable is the `this` pointer of the method `contains`. + +2. The restriction on parameter values +The meaning of `"p0":["..*"]` is: The constant string `..*` must be able to taint the parameter 0 of `contains` method. +3. NotTaint +This is the same as the TaintCheck format, the meaning is the opposite. It is required that these parameters and the base object of the method cannot be tainted by the source. + +An AND relationship exists between these two requirements, so: +```java +String path=zipEntry.getName(); +if(path.contains("../")){ + return false +} +File file=new File(path); +FileOutputStream fileOutputStream=new FileOutputStream(file); +``` +The path from variable `path` to variable `file` meets the rules, and thus is sanitized. + +On the contrary, a path in the below example cannot be sanitized. + +```java +String path=zipEntry.getName(); +String s=anotherFunction(); +if(s.contains("../")){ + return false; +} +if(path.contains("root")){ + return false; +} +File file=new File(path); +FileOutputStream fileOutputStream=new FileOutputStream(file); +``` +`s.contains("../")` meets the requirement check for p0, but not for TaintCheck. And `path.contains("root")` passes the requirement check for Taintcheck, but not for p0. So this propagation path is valid. + + +#### Sanitizer summary +Sanitizer is aimed at finding a path from source to sink, and then filtering it according to variables tainted by source. If the conditions are met, this path is removed, otherwise, it is retained. + +## The particularity of the four modes + +### Specificity of DirectMode +It needs to specify the entry of the analysis, such as +```json + "entry": { + "methods": [ + "" + ] + } +``` +The analysis entry for this rule is the `UnZipFolder` method. + +There can also be some dummy entries, but the purpose is the same, such as +```json + "entry": { + "ExportedCompos": true + } +``` +Each Android exported component is an analysis entry. For example, Activity's onCreate, onDestroy, etc. are all analysis entries. These entries are obtained by Appshark based on the parsing of the manifest file. They are not hard-coded, thus relatively flexible. Of course, for a specific app, you can analyze the manifest file manually, and then write the methods in each exported component into the rules to achieve the same effect. + +Another key field is `traceDepth`, which refers to how many layers of method calls are analyzed from the entry function. If the invocation level exceeds this depth, it will be ignored. + +### Specificity of SliceMode +The difference between SliceMode and DirectMode is that SliceMode's analysis entry is not fixed, but calculated according to the concrete source and sink. +The purpose of this mode is to deal with some scenarios: there is no fixed analysis entry; or the distance from the specified entry to the part of the code we want to analyze is too far, so it cannot be obtained the results in an acceptable time. + +How to calculate and analyze the entry according to source and sink, there are two main cases: +- Source is a parameter of a method +- Other forms of source +#### Source is a function parameter +Illustrate with the following example: +```json + "source": { + "Param": { + "": [ + "p1" + ] + } +} +``` +First, `shouldInterceptRequest` is a rule for a subclass of WebViewClient, because `android.webkit.WebViewClient` is an Android framework, we will not share the code of the framework directly. +The source here is the `p1` of `shouldInterceptRequest`, which is the parameter of `WebResourceRequest`. If we override the method `shouldInterceptRequest`, then we will start from this method, find all its explicitly or implicitly called functions, and check whether there is a method containing sink. If so, we use the overridden `shouldInterceptRequest` method as the analysis entry. + +There is a field `traceDepth` here, which refers to the number of method call layers tracked from `shouldInterceptRequest`. + +#### Other forms of source +Say, +```json +"source": { + "Return": [ + "" + ] +} +``` +It means that search downward from the `source` within the `traceDepth` layers, search upward from the `sink` within the `traceDepth` layers, and identify the method which is the first intersection of both directions as the analysis entry. + + +### ConstStringMode +It is special because there are too many constant strings in an app, so the entry of its analysis is not constrained by `traceDepth`, and the entry of its analysis is the method where the specified constant string locates. +Say, +```java +void f(){ + String s="constant_string" + g(s); +} + ``` + If `s` is a constant string satisfying our condition, then f() is the analysis entry. +The conditions for a constant string are: +- `constLen`: its length must be a multiple of this length +- `minLen`: its length cannot be less than this length +- `targetStringArr`: it formally satisfies any of these arrays, such as `"targetStringArr": ["AES","DES","*ECB*"]` + + +### ConstNumberMode +It is similar to ConstStringMode, and its analysis entry is also the method where the constant value locates. + +For a constant value condition, there is only `targetNumberArr`, which means that only the values in this array are concerned. +For example, +```json + "ConstNumberMode": true, + "targetNumberArr": [ + 16 + ] +``` + +## Advanced feature of rules + +### TaintTweak + +**Note that this section is only for non-whole program analysis mode.** + +In the mode of non-whole program analysis, if we want to block the spread of taint, or we know how to spread the taint, or the default rules cannot meet our requirements, then we can specify the taint propagation relationship for each rule individually. + +Say, + +```json +{ + "TaintTweak": { + "MethodSignature": { + "(java.lang.String,android.net.Uri,android.content.Context,java.lang.Class)>": {}, + }, + "MethodName": { + "putCharSequenceArrayListExtra": {} + } + } +} +``` + +The configuration of these rules is similar to the `VariableFlowRule` in `EngineConfig.json5`. `VariableFlowRule` is for all rules, while TaintTweak is for the current rule. + + +Then the meaning of the above rule is: +- The method `(java.lang.String,android.net.Uri,android.content.Context,java.lang.Class)>` will not do any taint propagation. +- A method `putCharSequenceArrayListExtra` also does not do any taint propagation. diff --git a/doc/en/overview.md b/doc/en/overview.md new file mode 100644 index 0000000..572e4c4 --- /dev/null +++ b/doc/en/overview.md @@ -0,0 +1,52 @@ +# Appshark introduction + +Appshark is a static analysis tool for Android apps. Its goal is to analyze very large apps (Douyin currently has 1.5 million methods). Appshark supports the following features: + +- JSON-based customized scanning rules to discover security vulnerabilities and privacy compliance issues you care about +- Flexible configuration -- you can tradeoff between accuracy and scanning time and space usage +- Custom extension rules to customize the analysis based on business needs + +## Overview + +Appshark is composed of modules: APK file processing, code preprocessing, user-defined rule analysis and analysis, pointer and data flow analysis, vulnerability identifying, sanitizer, and report generation. +See the complete flowchart ![Appshark workflow](appshark_flow_chart.png) + +### APK file preprocessing + +It mainly extracts the basic information of the app, such as exported components, manifest parsing, and discovering some common vulnerabilities in the manifest file. One of the most important tasks here is to use jadx to decompile the APK, and the generated java source code will be included in the final vulnerability details. + +### Code preprocessing + +There are three main functions of code preprocessing: + +1. Generate SSA +2. Generate the basic call graph +3. Patch various Jimple statements according to the configuration, such as callback injection. + +### User-defined rule parsing + +The main function of this module is to translate the fuzzy user-defined rules into concrete sources and sinks, and then based on the user's rule configuration, to find the relevant analysis entry, and to generate the `TaintAnalyzer`. The so-called `TaintAnalyzer` is a combination of source, sink, and entry. + +### Pointer and data flow analysis + +The input of this module is mainly an entry method. It also contains a series of user-defined or system-preset analysis rules. Through a long-time pointer analysis, `AnalyzeContext` is generated. +It contains the points-to and the data flow relationship obtained from the analysis starting from the specified entry. +The main idea of this module refers to the paper: [P/Taint: unified points-to and taint analysis](https://dl.acm.org/doi/10.1145/3133926) + +### Vulnerability identifying + +The input of this module mainly uses three parts: + +1. TaintAnalyzer: it finds the path from source to sink in the code +2. AnalyzeContext: it contains the data flow graph +3. Sanitizer in the association rules is used to filter out paths that do not meet the requirements. + +This module will look for the path from source to sink according to the data flow graph provided by `AnalyzeContext`. If a path is found and not filtered by Sanitizer, it will be added as a vulnerability in the final result. + +### Sanitizer + +The function of this module is to filter out the paths that do not meet the requirements according to the user-defined sanitizer. + +### Report generation module + +Each vulnerability will be displayed in a user-readable form. It will also output a `result.json` file, which contains all the vulnerability information. \ No newline at end of file diff --git a/doc/en/result.md b/doc/en/result.md new file mode 100644 index 0000000..aee361f --- /dev/null +++ b/doc/en/result.md @@ -0,0 +1,80 @@ +# Result interpretation + +Results.json is designed to facilitate program processing rather than human reading. We focus on the SecurityInfo and ComplianceInfo fields. + +## SecurityInfo + +The security vulnerabilities here will be classified according to the `category` and `name` specified by the `desc` in the rule. It is convenient for program processing and manual reading. +The `vulners` field is a list of vulnerabilities of this type. Each of these vulnerabilities has a hash field, which can be treated as a unique identifier for the vulnerability. +The details field contains a lot of information about the vulnerability: + +- Source: variables matching the source field in a rule +- Sink: variables matching the sink field in a rule +- position: the method where the source variable locates +- entryMethod: the entry of the analysis +- target: how the tainted data is propagated between variables +- url: use an HTML file to show how the tainted data is propagated between variables + +## ComplianceInfo + +ComplianceInfo is dedicated to privacy compliance issues. If the category is `ComplianceInfo`, then Appshark will process it specially. For example, + +```json +{ + "desc": { + "name": "GAID_NetworkTransfer_body", + "detail": "There exists obtaining operations sent over network-body", + "category": "ComplianceInfo", + "complianceCategory": "PersonalDeviceInformation_NetworkTransfer", + "complianceCategoryDetail": "PersonalDeviceInformation_NetworkTransfer", + "level": "3" + } +} +``` + +Its classification is: + +- The 1st level is ComplianceInfo +- The 2nd level is PersonalDeviceInformation_NetworkTransfer specified by ComplianceCategory +- The 3rd level is GAID_NetworkTransfer_body specified by name. + +Another example: + +```json +{ + "ComplianceInfo": { + "PersonalDeviceInformation_NetworkTransfer": { + "GAID_NetworkTransfer_body": { + "category": "ComplianceInfo", + "detail": "There exists obtaining operations sent over network-body", + "name": "GAID_NetworkTransfer_body", + "vulners": [], + "deobfApk": "", + "level": "3" + } + } + } +} +``` + +As for the field of vulners, its meaning is the same as that in SecurityInfo. + +## Vulnerability details page introduction + +The purpose of the vulnerability details page is designed to display information to users independently of the results.json file, so as to facilitate the root cause analysis of the vulnerability. + +### vulnerability detail + +It is the basic information of the app and its vulnerability. + +### data flow + +The target field mentioned above + +### call stack + +Methods that are involved in taint propagation. + +### code detail + +Show the process of taint propagation in detail. If javaSource is true in `config.json5`, the java code of the decompiled methods is also displayed. \ No newline at end of file diff --git a/doc/en/startup.md b/doc/en/startup.md new file mode 100644 index 0000000..7fdb7cb --- /dev/null +++ b/doc/en/startup.md @@ -0,0 +1,27 @@ +How to get started quickly, take a simple vulnerability scan as an example + +# 1. Download jar dependencies + +Maven Repository provides the complete list of jar files that can be downloaded [](). Install jre/jdk 11 for our engine. + +# 2. Download the config folder on Github + +```shell +git clone +``` + +# 3. Update the config file + +1. Change apkPath to the target apk file's absolute path. +2. Specify the rule(s) to be applied, separated by a comma. The rules should locate in the config/rules folder, since Appshark searches this folder for the rules. +3. Specify the output folder of the results. Its default value is ./out folder, you can change it to any other directory. + +# 4. Run Appshark + +```shell +java -jar AppShark-0.1-all.jar config/config.json5 +``` + +# 5. Check out results +The results locate in the output folder (./out by default). The results.json file gives the detected vulnerability list. A detailed explanation can be found in [](result.md). +If you have questions about a specific vulnerability, you can check the file provided by the url field. \ No newline at end of file diff --git a/doc/zh/EngineConfig.md b/doc/zh/EngineConfig.md new file mode 100644 index 0000000..da0f44e --- /dev/null +++ b/doc/zh/EngineConfig.md @@ -0,0 +1,137 @@ +# EngineConfig 说明 +EngineConfig.json5 是对引擎更深入的配置,如果不理解其准确含义,建议不要修改这个文件. + +## IgnoreList +被这里命中的函数在分析时,将会被忽略. 指定函数的规则有三种形式: +- PackageName 整个包下面的函数都会被忽略 +- MethodName 以函数名字进行匹配 +- MethodSignature 以函数签名进行匹配. + +## Callback +当创建碰到指定的类被创建时,自动调用相关函数. 比如 +```json + "java.lang.Runnable": [ + "void run()" + ] +``` +```java +Runnable r=new Runnable(){ + @Override + public void run() { + //do something + } +}; +//相当于自动在这条语句下面添加了一条 +//r.run(); +``` + +## Library + +指定的package,将会被认为是library,主要用于降低Appshark分析的工作量,因为通常认为framework,jdk等代码中不会存在漏洞. +如果一个package被标记为Library,主要有一下几点需要注意: +1. 构建call graph的时候会自动忽略library的代码 +2. source,sink中会有对library的限制 +3. library的代码在分析的时候,会通过PointerFlowRule和VariableFlowRule计算指针以及数据流传播规则,而不是根据其具体实现代码. + + +## PointerFlowRule +**如果不确定要不要修改该配置,那么就不要修改,除非有明确的证据** +该配置主要是解决library代码以及超出规则中指定的`traceDepth`时,如何处理相关的函数调用的指针指向关系. +PointerFlowRule 一个基本原则是: **如果没有匹配的规则,那么就不处理相关的指向关系**. + +该规则中的关键字: +- p* 所有参数 +- @this 函数调用时的this指针 +- ret 函数的返回值 +- p0,p1,p2 函数的参数0,1,2,... +- @this.data 一个虚拟字段,代表this对象的所有字段. +- I 表示输入 +- O 表示输出 + +MethodName 是按照函数名来匹配函数,MethodSig是按照函数签名来匹配函数. + +以keySet规则为例来说明: +```json + "": { + "@this.data->ret": { + "I": [ + "@this.data" + ], + "O": [ + "ret" + ] + } + } +``` +碰到如下代码: +```java +HashMap m=new HashMap(); +m.put("s1","s2"); +m.put("i",new Intent()); +Set s= m.keySet(); +``` +变量s(也就是keySet的返回值)将指向m中的所有字段. 如果此时`@this.data`指向了`["s1","s2","i",new Intent()]`这些object,那么s也会指向这些object. +但是s并不会指向`new HashMap()` 这个对象,因为没有从this到ret的关系. + +## VariableFlowRule + +该配置主要是解决library代码以及超出规则中指定的`traceDepth`时,如何处理相关的函数调用的数据流向关系. +它与PointerFlowRule在处理时的一个重要区别是,它有缺省的数据流向关系,也就是: +- InstantDefault +- InstantSelfDefault + +InstantSelfDefault和InstantDefault的区别是,前者明确要求caller和callee的this指针是相同的. 比如: +```java +class C{ + void f(){ + g(); + } + void g(){ + + } +} +``` +那么在分析f中对g的调用时,数据流向关系按照InstantSelfDefault而不是InstantDefault. + +**VariableFlowRule的关键字和PointerFlowRule是一样的** + +- p* 所有参数 +- @this 函数调用时的this指针 +- ret 函数的返回值 +- p0,p1,p2 函数的参数0,1,2,... +- @this.data 一个虚拟字段,代表this对象的所有字段. +- I 表示输入 +- O 表示输出 + +以下面规则为例来解释: +```json +"": { +"@this.data->ret": { + "I": [ + "@this.data" + ], + "O": [ + "ret" + ] +}, +"@this->ret": { + "I": [ + "@this" + ], + "O": [ + "ret" + ] +} +} +``` + +```java +HashMap m=new HashMap(); +m.put("s1","s2"); +m.put("i",new Intent()); +Object obj= m.remove("i3"); +``` +如果此时`@this.data`指向了`["s1","s2","i",new Intent()]`这些object,那么HashMap数据将会从`["s1","s2","i",new Intent()]`流向m,进而会从m流向obj. + +也就是,如果`new HashMap()`是污点,那么obj将会被污染. 如果"s1"是污点,那么obj也会被污染. + diff --git a/doc/zh/appshark_flow_chart.jpg b/doc/zh/appshark_flow_chart.jpg new file mode 100644 index 0000000..e0ba16b Binary files /dev/null and b/doc/zh/appshark_flow_chart.jpg differ diff --git a/doc/zh/argument.md b/doc/zh/argument.md new file mode 100644 index 0000000..700db4b --- /dev/null +++ b/doc/zh/argument.md @@ -0,0 +1,169 @@ +# 运行参数说明 + +运行参数配置和EngineConfig都是对appshark的参数调整,但是前者针对的是本次app的扫描,而后者相对稳定,针对的是所有的App. + +## 必须指定的参数 + +apkPath: 要扫描的app的绝对路径. +rules: 扫描使用的规则,以逗号分隔的一系列规则列表. + +## 其他缺省参数 + +### CallBackEnhance + +CallBackEnhance, 如果该参数为true,那么默认会把所有的匿名类重写的函数直接进行调用. 比如: + + ``` + sensitiveApi.setOnClickListener(new MyListener() { + public void onClick(View v) { + //call + } + }); + //会在下面插入一行 + // l.onClick(null); //其中l是刚刚创建的MyListener + ``` + +### maxPointerAnalyzeTime + +指针分析的超时时间,以秒为单位. 如果该时间较短,可能会导致因为部分代码没有分析到,而导致误报和漏报. + +### javaSource + +是否在最终的漏洞详情中展示源码. 该源码是通过jadx反编译得到. + +### maxThread + +控制内部进行指针分析等操作时的并行度,默认数量为本机cpu的核数. 如果发生了OOM,可以通过降低该数值来节省内存,但是这会导致分析时间的增加. + +### rulePath + +规则文件所在的文件夹,请以绝对路径给出. 默认是当前工作目录下的`lib/rules`. + +### sdkPath + +安卓框架所在目录,默认是当前工作目录下的`tools/platforms` + +### toolsPath + +jadx,platforms所在的目录,默认为当前工作目录下的`tools`. + +### supportFragment + +是否对处理Fragment的lifeCycle函数. 类似于处理Activity的onCreate等函数. + +### logLevel + +日志输出级别,默认为info (1): + +- debug 0 +- info 1 +- warn 2 +- error 3 + +### ruleMaxAnalyzer + +防止出错用,如果一个规则写的有问题,可能会导致匹配到非常多的source和sink. 这个数值默认5000,绝大多数情况下都应该满足要求, +如果你在分析时,碰到`rule xxxx has too many rules: xxxx, dropped`,可以考虑增加这个数值. + +### maxPathLength + +source到sink的最长路径,超过这个长度的路径,就算是有效也会被丢弃. + +### wholeProcessMode + +是否进行全程序分析,默认为false. 全行程分析主要是影响分析的范围. +开启全程序的影响 + +- 可能会大幅加大分析时间. 该选项会导致Appshark尽可能的分析app中的每一条指令,而不关心他与用户指定的source,sink的关系. +- 因为分析范围的增加,可能会带来更多的误报,也可能会带来更多的发现. +- 指针分析过程将会是严格的单线程,无法并行 + +#### 什么时候建议开启该选项 + +如果你分析的app比较小,能够在你可接受的时间空间之内取得结果,那么建议开启此选项. + +#### 什么时候不应该开启该选项 + +如果你的app非常大, 代码非常复杂,那么你不应该开启次选项. 因为他几乎不可能在你可接受的时间之内分析出结果. + +#### 与SliceMode,DirectMode的关系 + +在具体的Rule中,无论是通过直接还是间接方式,都会有一个分析入口以及分析的深度(traceDepth),但是一旦开启了全程序分析,那么这些入口和深度都将会被视若无睹. +当然指针分析完成之后的路径查找,仍然会使用相关Rule中的source,sink以及sanitizer. + +#### skipAnalyzeNonRelatedMethods + +从分析入口开始,在指定的`traceDepth`下,可以抵达很多函数,那么是否有必要对所有这些函数都进行分析呢? 有一个选择是,只对和source,sink相关的函数进行分析. +默认为false,即对这些所有函数进行分析,为true,则跳过那些明显与source,sink无关的函数. +比如: + +``` +Class C{ + void f(){ + Object o=source(); + g(o); + h(); + } + void g(Object o){ + sink(o); + } + void h(){ + + } +} +``` + +如果f为分析的入口,skipAnalyzeNonRelatedMethods为true的时候,会跳过对h的分析,就像`h()`调用不存在一样. 否则会对h进行分析. + +#### skipPointerPropagationForLibraryMethod + +目前的指针分析过程采用的是上下文不明感分析(context insensitive),对于像`java.util.Iterator: java.lang.Object next()`这样的函数,就会大量普遍存在. +实际上普通地方的next函数的this指针以及返回值都是没有关系的. 因此在这种场景下,如果避免做指针的propagation,可以节省不少时间. 当然不propagation有可能会带来误报和漏报. + +该值默认为true,即不对library中的method做指针的propagation,只按照规则做一次指向计算. + +```java +Class C{ + void f(){ + StringBuffer s1=new StringBuffer(); + StringBuffer s2=null; + StringBuffer s3=s2; + s2=s1.append("aa"); + String s=s3.toString(); + } +} +``` +注意EngineConfig.json5中有这样的规则: +```json5 +{ + PointerFlowRule: { + "MethodSignature": { + "": { + "@this->ret": { + "I": [ + "@this" + ], + "O": [ + "ret" + ] + } + }, + }, + } +} +``` +如果skipPointerPropagationForLibraryMethod为true,append是library中的method,所以在指针传播过程中,会根据规则,得到 +s1->{new StringBuffer();} //根据 StringBuffer s1=new StringBuffer(); +s2->{new StringBuffer();,null} //根据s2=s1.append("aa"); 以及PointerFlowRule +s3->{null} + +如果skipPointerPropagationForLibraryMethod为false,那么指针指向关系将是: +s1->{new StringBuffer();} //根据 StringBuffer s1=new StringBuffer(); +s2->{new StringBuffer();,null} //根据s2=s1.append("aa"); 以及PointerFlowRule +s3->{new StringBuffer();,null} + + + + + + diff --git a/doc/zh/faq.md b/doc/zh/faq.md new file mode 100644 index 0000000..a0453fd --- /dev/null +++ b/doc/zh/faq.md @@ -0,0 +1,44 @@ +# 常见问题 + +## 1. Appshark与soot的关系? +Appshark分析的对象是soot生成的jimple IR,soot 是Appshark的基座. 但是Appshark有自己独立的规则系统,以及独立实现的指针分析和污点分析算法. + +## 2. Appshark与flowdroid的关系? +Appshark使用了flowdroid对android manifest,xml文件,各种resouce文件等的解析模块,但是Appshark有自己独立的规则系统,以及独立实现的指针分析和污点分析算法. + +## 3. sliceMode 可能存在什么问题? + +```java +public class SliceMode { + public static void main(String[] args) { + A x=new A(); + A y=x; + f(x,y); + } + void f(A x,A y){ + dosource(x); + dosink(y); + } + void dosource(A x){ + x.f=source(); + } + void dosink(A y){ + sink(y.f); + } +} +``` +如果sliceMode进行分析,那么会得到分析的入口点是函数f, 那么将无法得到x,y实际上指向的是同一个object,从而无法形成从source到sink的传播路径,导致漏报. + +## 4. field sensitive +我们的分析是field sensitive的,不会混淆不同field的传播. +比如: +```java +public class FieldSensitive{ + public static void main(){ + A x=new A(); + x.f1=source(); + sink(x.f2); + } +} +``` +不会形成一条从source到sink的传播路径,因为x.f1和x.f2是不同的field. \ No newline at end of file diff --git a/doc/zh/how_to_find_compliance_problem_use_appshark.md b/doc/zh/how_to_find_compliance_problem_use_appshark.md new file mode 100644 index 0000000..a50c37f --- /dev/null +++ b/doc/zh/how_to_find_compliance_problem_use_appshark.md @@ -0,0 +1,68 @@ +# 如何使用appshark找出程序中的隐私合规问题. + +Appshark可以对应用进行数据流分析，也可以只查找某个API的调用点，这两种功能都可以帮助你分析应用中可能存在的隐私合规风险。隐私合规相关规则的编写方式与安全漏洞规则基本一致，因此在开始之前，请先阅读 如何为Appshark撰写规则 +## 隐私数据流分析 +隐私数据流分析就是数据流分析的其中一种，但绝大部分时候你不需要编写entry和sanitizer，你应该更关心规则中的source和sink。 +具体来说，你可以将source指定为某个获取隐私信息的API，例如： +``` +"source": { + "Return": [ + "" + ] +} +``` +这个API的return是设备唯一标识IMEI。 +同时，将sink指定为你认为会存在隐私数据泄露的方法，例如写文件： +``` +"sink": { + "": { + "TaintCheck": [ + "p0" + ] + } +} +``` +当你所关注的隐私数据来源并非API，而是对象的某个field时，你依然可以按照通用规则编写中field类型source的格式编写规则，例如设备序列号作为source： +``` +"source": { + "Field": [ + "" + ] +} +``` +最后，你需要使用SliceMode，可以减少分析的时间。 + + +完整的规则文件: +```json +{ + "getDeviceId": { + "SliceMode": true, + "traceDepth": 8, + "desc": { + "name": "getDeviceId", + "category": "ComplianceInfo", + "detail": "", + "wiki": "", + "complianceCategory": "PersonalDeviceInformation", + "complianceCategoryDetail": "PersonalDeviceInformation", + "level": "3" + }, + "source": { + "Return": [ + "" + ], + "Field": [ + "" + ] + }, + "sink": { + "": { + "TaintCheck": [ + "p0" + ] + } + } + } +} +``` \ No newline at end of file diff --git a/doc/zh/how_to_write_rules.md b/doc/zh/how_to_write_rules.md new file mode 100644 index 0000000..885a2f2 --- /dev/null +++ b/doc/zh/how_to_write_rules.md @@ -0,0 +1,463 @@ +# 如何为Appshark撰写规则 + +使用Appshark进行数据流分析,最重要的就是明确告诉Appshark你关心的分析入口,source,sink以及santizer. 根据source的特殊性,将其分类为: + +- ConstStringMode 支持常量字符串作为source +- ConstNumberMode 支持常量整数作为source +- SliceMode和DirectMode 其他类型的source + +## 背景知识 + +Appshark分析的对象是经过SSA处理的jimple指令,因此在指定source/sink的时候,引用的函数以及field签名必须符合jimple格式. + +### jimple函数签名 + +`` +这是一个通用的Java函数签名,包含了类名,函数名,函数返回类型,参数类型列表. 在指定source,sink的过程中,每个部分都可以用*来模糊匹配. +比如`<*: android.content.Intent parseUri(java.lang.String,int)>` 匹配所有类中,函数名字为`parseUri` +,返回类型是Intent以及参数列表为`java.lang.String,int`的函数. + +### jimple field签名 + +`` +这是一个通用的Java 对象的field签名,这个field的类名是`com.security.TestClass`,类型是`android.content.Intent`,field的名字是`fieldName`. +**field签名不支持模糊匹配指定,必须准确给出** + +## 一般规则的撰写 + +一般规则包含四个部分,分别是:1. 分析的入口 2. source 3. sink 4. sanitizer. + +### 分析入口的指定 + +分析的入口一般是一个函数. 比如 + +```json +"entry": { +"methods": [ +"" +] +} +``` + +entry只有在`DirectMode`下需要明确指定,其他三个模式下,都无需明确指明分析入口. 如果你不知道分析入口是什么,说明你不应该使用`DirectMode`. + +### 一般source的指定 +**需要说明的是,appshark内部真正的source点会是具体的变量,因此无论哪种写法,都会转换成一个具体的变量.** +source可以有很多种类型,分别是: + +- 常量字符串,注意这与`ConstStringMode`是没关系的 +- 函数返回值 +- 函数的某个参数 +- 对象的某个field +- 某个对象的创建 + + +下面分别举例介绍这五种情况. + +#### 常量字符串 + +```json +"source": { +"ConstString": ["path1"] +} +``` + +那么: + +```java +String s="path1"; +f(12,"path1"); +``` + +s将成为source. +函数f的参数1将成为source. + +#### 函数的返回值 + +这种一种最常见的source形式,比如: + +```java +"source": { + "Return": [ + "" + ] +} +``` +也就是getName的返回值将会是source. +那么: +```java +ZipEntry e=getEntry(); +String name=e.getName(); +``` +name将成为source点. + +#### 某对象的field +比如: +```json + "source": { + "Field": [ + "", + ] + } +``` +那么: +```java +Uri uri=CalendarContract.CONTENT_URI; +``` +uri将会成为source点. +**注意不区分该field是静态field还是非静态field** + +#### 某个函数的参数 +函数参数作为source一般在重写系统类的情况, +比如: +```json + "source": { + "Param": { + "": [ + "p1" + ] + } +} +``` +首先注意,p0是第一个参数,p1是第二个参数,这里类型为WebResourceRequest才是source. + +#### 某个对象的创建 +这个规则非常特殊,一般不会用到. +比如: +```json + "source": { + "NewInstance": ["android.content.Intent"] + } +``` +那么: +```java +android.content.Intent i=new android.content.Intent(); +``` +这时候变量i将成为source点. + +### 一般sink的指定 +目前sink点只能是函数的参数,可以是: +- this指针 @this +- 函数的某个参数 p0,p1,p2 +- 函数的所有参数 p* + +#### sink +需要强调的是,**所有的sink都会在内部转换成具体的变量**. +sink的指定相对于source的指定要简单许多,种类也比较单一. +例如: +```json + "sink": { + "(*)>": { + "LibraryOnly":true, + "TaintCheck": [ + "p*" + ] + }, + "(*)>": { + "TaintCheck": [ + "p*","@this" + ] + } + } +``` + 那么: + ```java +String path; +File f=new File(path); +FileOutputStream fileOutputStream=new FileOutputStream(f); + ``` + 这里面的f,fileOutputStream都会是sink点. appshark会检查能否找到从source到这些变量的一个污点传播路径. + +sink还有一个可配置的选项就是`LibraryOnly`,默认值为false,如果设置为true,那么就要求匹配到的函数签名必须是`EngineConfig.json5`中指定的Library. +以上面例子为例,如果在`EngineConfig.json5`中指定`com.security`为Library,那么path就是sink点. +否则如果没有在`EngineConfig.json5`中指定`com.security`为Library,那么path就不是sink点. + +### sanitizer的指定 + +sanitizer目的是消除误报. 虽然发现了一条从source到sink的完整传播路径,但是因为已经对source做了严格的校验,所以这并不是一条有效的路径. +下面以unzipSlip规则为例来介绍一下sanitizer的原理. +zip slip漏洞的原理可以参考[Directory traversal attack](https://en.wikipedia.org/wiki/Directory_traversal_attack). 主要是在解压zip文件的时候,没有检查文件名中是否包含"../",导致如果zip文件外部可控的话,可能会导致任意文件覆盖问题. + +首先给出完整的规则: +```json +{ + "unZipSlip": { + "SliceMode": true, + "traceDepth": 8, + "desc": { + "name": "unZipSlip", + "category": "FileRisk", + "detail": "ZIP Slip is a highly critical security vulnerability aimed at these kinds of applications. ZIP Slip makes your application vulnerable to Path traversal attack and Sensitive data exposure.", + "wiki": "", + "possibility": "4", + "model": "middle" + }, + "source": { + "Return": [ + "" + ] + }, + "sanitize": { + "getCanonicalPath": { + "": { + "TaintCheck": [ + "@this" + ] + } + }, + "containsDotdot": { + "": { + "TaintCheck": [ + "@this" + ], + "p0": [ + "..*" + ] + } + }, + "indexDotdot": { + "": { + "TaintCheck": [ + "@this" + ], + "p0": [ + "..*" + ] + } + } + }, + "sink": { + "(*)>": { + "TaintCheck": [ + "p*" + ] + }, + "(*)>": { + "TaintCheck": [ + "p*", + "@this" + ] + } + } + } +} +``` + +source和sink就不展开说了,上面刚刚介绍过. + +重点说一下sanitizer,因为它的设计不是那么容易理解. + +#### 顶层规则是或的关系 +sanitizer分别包含了三个子key: +- getCanonicalPath +- containsDotdot +- indexDotdot +这三个规则是或的关系. 根据规则,我们会找到N个 source,M个sink. 那么理论上就会存在N*M条路径.对于其中的任意一条路径,如果它满足了这三条规则中的任意一条,就会被sanitize掉. + +#### 二层规则之间是与的关系 +由于这个例子中,二层规则都只有单独一条,所以这里造一个规则来演示. +```json + "containsDotdot": { + "": { + "TaintCheck": [ + "@this" + ], + "p0": [ + "..*" + ] + }, + "": { + "TaintCheck": [ + "@this" + ] + } + } +``` +如果某条路径同时满足对``和``这两个函数的限制,那么这条路径就会被santize掉. + +#### 具体规则的含义 +再次强调,**appshark分析的是污点在变量之间的传递关系,所以无论是source,还是sink,还是sanitizer描述的具体粒度都是变量.** +以containsDotdot为例: +```json +"containsDotdot": { + "": { + "TaintCheck": [ + "@this" + ], + "p0": [ + "..*" + ] + } +} +``` +它的限制有两个: +1. TaintCheck +从source出发,传播到的所有变量中,是否污染到了``这个函数的this指针. +比如: +```java +if(path.contains("../")){ + return false +} +``` +那么这里的path就是contains的this指针, + +2. 参数取值的限制 +`"p0":["..*"]`的含义是: 常量字符串`..*`要能污染到contains的参数0. +3. NotTaint +这个和TaintCheck格式一样,意思相反,要求函数的这些地方不能被source污染到. + +这两个条件之间也是与的关系,因此: +```java +String path=zipEntry.getName(); +if(path.contains("../")){ + return false +} +File file=new File(path); +FileOutputStream fileOutputStream=new FileOutputStream(file); +``` +满足我们的sanitizer,从source(path)到sink(file)的这条传播路径就会被sanitize掉. + +反之: +下面的例子中,就不能被sanitize掉. + +```java +String path=zipEntry.getName(); +String s=anotherFunction(); +if(s.contains("../")){ + return false; +} +if(path.contains("root")){ + return false; +} +File file=new File(path); +FileOutputStream fileOutputStream=new FileOutputStream(file); +``` +这个例子中`s.contains("../")`满足了对p0的检验,但是没有满足TaintCheck的检验. 而`path.contains("root")`满足了对Taintcheck的检验,但是没有满足对p0的检验. 所以这条传播路径是有效的. + + +#### sanitizer总结 +sanitizer针对的是,已经找到了一条从source到sink的路径,再根据source污染到的所有变量进行过滤,如果满足条件就删掉这条路径,否则保留. + +## 四种mode的特殊性 + +### DirectMode的特殊性 +它需要明确指明分析的入口,比如: +```json + "entry": { + "methods": [ + "" + ] + } +``` +那么这条规则的分析入口就是`UnZipFolder`这个函数. + +这里还可以是一些虚拟入口,但是目的都是一样的,比如: +```json + "entry": { + "ExportedCompos": true + } +``` +这里的意思是,每个安卓的导出组件都是分析入口. 比如Activity的onCreate,onDestroy等等都是分析的入口. 这些入口有appshark根据manifest文件的解析得到,而不是写死,相对灵活一点. 当然你也可以针对具体的app,自行分析manifest文件,然后把每个导出的组件中的函数写到规则中,这样效果是一样的. + +这里还有一个关键特性就是`traceDepth`,这里指的是从分析入口函数开始,分析多少层函数调用为止. 如果调用层级超过这个深度,会被忽略. +### SliceMode的特殊性 +SliceMode和DirectMode的区别是它的分析入口不是固定的,而是根据具体的source,sink计算得到的. +这个mode的提出针对的是,在某些场景下,就没有固定的分析入口. 或者从指定的入口开始到我们想要分析的那部分代码之间距离太远,导致不能在有效的时间内取得分析结果. + +怎么根据source和sink计算分析入口,主要有两种情况: +- source为某个函数的参数 +- 其他形式的source +#### source为某个函数参数 +以下面的例子来说明: +```json + "source": { + "Param": { + "": [ + "p1" + ] + } +} +``` +首先shouldInterceptRequest针对是WebViewClient的子类而言的,因为`android.webkit.WebViewClient`是安卓的framework,我们并不会直接去分享framework的代码. +这里的source是shouldInterceptRequest的p1,也就是`WebResourceRequest`这个参数. 如果我们override了`shouldInterceptRequest`这个函数,那么将会从这个函数出发,找出它所有的显式或者隐式的被调函数,看看里面有没有包含sink点的函数. 如果有就将这个override的`shouldInterceptRequest`函数作为分析入口. + +这里有一个规则的`traceDepth`,指的是从shouldInterceptRequest出发,查找的函数层数. + +#### 其他形式的source +比如: +```json +"source": { + "Return": [ + "" + ] +} +``` +意思是从`source`点往下搜索`traceDepth`层,从`sink`点往上搜索搜索`traceDepth`层,找到它们最近交汇的函数作为分析入口. + + +### ConstStringMode +它之所以特殊,因为app中的常量字符串太多可能非常多,所以其分析的入口不受`traceDepth`的约束,它分析的入口就是指定的常量字符串所在的函数. +比如: + ```java +void f(){ + String s="constant_string" + g(s); +} + ``` + 如果s是满足我们条件的常量字符串,那么f就是分析入口. + 限制常量字符串的条件有: + - constLen 长度必须是这个长度的倍数 + - minLen 长度不能小于这个长度 + - targetStringArr 形式上满足这个数组中的任意一个,比如 `"targetStringArr": ["AES","DES","*ECB*"]` + + + ### ConstNumberMode + 它与ConstStringMode类似,其分析入口也是这个常量数值所在的函数. + +对于常量数值的限制,只有`targetNumberArr`,它表示只关心这个数组里面的数值. +比如: +```json + "ConstNumberMode": true, + "targetNumberArr": [ + 16 + ] +``` + + + +## 规则的高级特性 + +### TaintTweak + +**注意这部分内容只针对非全程序分析模式。** + +在非全程序分析的模式下,如果我们想阻断污点的传播,或者我们自己知道污点怎么传播,默认的规则不能满足我们的要求,可以为单独每个规则指定污点传播关系. + +比如 + +```json +{ + "TaintTweak": { + "MethodSignature": { + "(java.lang.String,android.net.Uri,android.content.Context,java.lang.Class)>": {}, + }, + "MethodName": { + "putCharSequenceArrayListExtra": {} + } + } +} +``` + + + +这部分规则的配置方式和`EngineConfig.json5`中的`VariableFlowRule`的含义是类似的,只不过`VariableFlowRule`针对的是所有的规则,而TaintTweak针对的是当前这一条规则. + + + +那么上面的规则的含义就是: + +- 函数`(java.lang.String,android.net.Uri,android.content.Context,java.lang.Class)>` 将不做任何污点传播. +- 名为`putCharSequenceArrayListExtra`的函数也不会做任何污点传播. + + + diff --git a/doc/zh/overview.md b/doc/zh/overview.md new file mode 100644 index 0000000..09444fd --- /dev/null +++ b/doc/zh/overview.md @@ -0,0 +1,56 @@ +# Appshark 介绍 + +Appshark 是一个针对安卓的静态分析工具,它的设计目标是针对超大型App的分析(抖音目前已经有150万个函数). Appshark支持众多特性: + +- 基于json的自定义扫描规则,发现自己关心的安全漏洞以及隐私合规问题 +- 灵活配置,可以在准确率以及扫描时间空间之间寻求平衡 +- 支持自定义扩展规则,根据自己的业务需要,进行定制分析 + +## 结构 + +Appshark有apk文件处理,代码预处理, 用户自定义规则分析解析,指针以及数据流分析,漏洞查找, sanitizer以及报告生成等模块组成. +完整的流程图见![appshark工作流程](appshark_flow_chart.jpg) +流程图原图在这里: +https://bytedance.feishu.cn/docx/doxcni8LJ4cSfiTejoSnf9I7tKh + +### apk文件预处理 + +主要是提取app中的基本信息,比如导出组件,manifest解析,以及发现一些manifest中常见的漏洞. 这里面还有一个最重要的工作就是会使用jadx对apk进行反编译,其生成的java源码会在最后的漏洞详情中展示. + +### 代码预处理 + +代码预处理最主要有三个功能: + +1. 生成SSA +2. 生成基本的call graph +3. 根据配置进行各种指令的patch,比如callback注入. + +### 用户自定义规则解析 + +该部分主要的功能就是将模糊的用户自定义规则翻译为准确的source以及sink,然后根据用户的规则配置,查找相关的分析入口,生成`TaintAnalyzer`. 所谓的`TaintAnalyzer` +就是source,sink,entry的一个综合体. + +### 指针以及数据流分析 + +该模块的输入主要是一个所谓的入口函数,当然也包含了一系列用户自定义的或者系统预置的分析规则. 通过较长时间的指针分析,生成`AnalyzeContext`, `AnalyzeContext` +里面包含了从指定的入口分析以后,得到的指针指向关系以及数据流流向关系. +该模块的主要思想主要是参考了论文: [P/Taint: unified points-to and taint analysis](https://dl.acm.org/doi/10.1145/3133926) + +### 漏洞查找 + +该模块的输入主要用三部分: + +1. TaintAnalyzer,查找其中的source到sink的路径 +2. AnalyzeContext, 包含了数据流图 +3. 关联规则中的Sanitizer,用于过滤掉不符合要求的路径. + +该模块会依据`AnalyzeContext`提供的数据流图,查找从source到sink的路径,如果找到,并且该路径没有被Sanitizer过滤掉,那么就会在最终结果中添加一个漏洞. + +### Sanitizer + +该模块的功能就是根据用户自定的sanitizer,过滤掉不符合要求的路径. + +### 报告生成模块 + +每个漏洞会以用户可以阅读的方式进行展示. 同时会给一个`result.json`,这里面包含了所有的漏洞信息. + diff --git a/doc/zh/path_traversal_game.md b/doc/zh/path_traversal_game.md new file mode 100644 index 0000000..f40c3d0 --- /dev/null +++ b/doc/zh/path_traversal_game.md @@ -0,0 +1,459 @@ +# appshark深入教程 + +以一个path traversal的游戏贯穿本教程,让大家体会一下如何发现漏洞,修复漏洞以及如何用appshark发现问题. + +## 1. 什么是目录遍历漏洞 +根据维基百科定义: +目录遍历（英文：Directory traversal），又名路径遍历（英文：Path traversal）是一种利用网站的安全验证缺陷或用户请求验证缺陷（如传递特定字符串至文件应用程序接口）来列出服务器目录的漏洞利用方式。 +此攻击手段的目的是利用存在缺陷的应用程序来获得目标文件系统上的非授权访问权限。与利用程序漏洞的手段相比，这一手段缺乏安全性（因为程序运行逻辑正确）。 +目录遍历在英文世界里又名../ 攻击（Dot dot slash attack）、目录攀登（Directory climbing）及回溯（Backtracking）。其部分攻击手段也可划分为规范化攻击（Canonicalization attack）。 + +## case1: 无任何校验 +我们的app有一个content provider,用来共享sandbox目录下的文件. 简单实现如下: +```manifest + +``` + +对应的provider是: +```java + +public class VulProvider1 extends ContentProvider { + + @Nullable + @Override + public ParcelFileDescriptor openFile(@NonNull Uri uri, @NonNull String mode) throws FileNotFoundException { + File root = getContext().getExternalFilesDir("sandbox"); + String path = uri.getQueryParameter("path"); + return ParcelFileDescriptor.open(new File(root, path), ParcelFileDescriptor.MODE_READ_ONLY); + } +} +``` + +你能发现找到其中的问题么? 你能绕过限制,读取到`/data/data/com.security.bypasspathtraversal/files/file2`文件么? + +### 如何目录遍历呢? +作者的意图是只共享sandbox目录,但是他直接把用户path作为参数传递给了File,这意味着,如果path中包含"../",那么就可以绕过sandbox目录限制. +可以轻松构造出一个poc: +```java +String path="content://slipme1/?path=../../../../../../../../data/data/com.security.bypasspathtraversal/files/file2"; + String data = IOUtils.toString(getContentResolver().openInputStream(Uri.parse(path))); +``` +### 如何利用appshark发现此类漏洞 +那么,如何利用利用appshark来自动发现此类漏洞呢? 关键就是定义source,sink以及sanitizer. +明显openFile的参数0也就是uri是用户可控制的,一般把外部用户可直接或间接控制的变量视为source. 而sink点比较合适的一个地方是`ParcelFileDescriptor.open`的参数0, +因为如果source能够控制`ParcelFileDescriptor.open`参数0,那么基本上就可以读取任何文件了. + +因此source,sink定义如下: +```json +{ + "source": { + "Param": { + "<*: android.os.ParcelFileDescriptor openFile(*)>": [ + "p0" + ] + } + }, + "sink": { + "": { + "TaintCheck": [ + "p0" + ] + } + } +} +``` +### 完整的规则 +```json +{ + "ContentProviderPathTraversal": { + "enable": true, + "SliceMode": true, + "traceDepth": 14, + "desc": { + "name": "ContentProviderPathTraversal", + "category": "", + "wiki": "", + "detail": "如果Content Provider重写了openFile，但是没有对Uri进行路径合法性校验，那么攻击者可能通过在uri中插入../的方式访问预期外的文件", + "possibility": "", + "model": "" + }, + "source": { + "Param": { + "<*: android.os.ParcelFileDescriptor openFile(*)>": [ + "p0" + ] + } + }, + "sink": { + "": { + "TaintCheck": [ + "p0" + ] + } + } + } +} +``` + +### 验证 + +app完整的源码位于 [BypassPathTraversal](https://github.com/nkbai/BypassPathTraversal). apk文件也在这个repo中[下载apk](https://github.com/nkbai/BypassPathTraversal/blob/main/apk/app-debug.apk). +完整的config文件: +```json +{ + //apk to anlayze + "apkPath": "/Users/bai/Downloads/traversal/BypassPathTraversal/app/build/outputs/apk/debug/app-debug.apk", + //result output directory + "out": "out", + "rules": "ContentProviderPathTraversal.json", + "maxPointerAnalyzeTime": 600 +} +``` +运行命令如下: +```shell +java -jar AppShark-0.1-all.jar config/config.json5 +``` + +可以在out目录中的results.json中发现下面的内容: +```json +{ + "details": { + "Sink": [ + "->$r5" + ], + "position": "", + "Manifest": { + "exported": true, + "trace": [ + "" + ], + "": [ + ] + }, + "entryMethod": "", + "Source": [ + "->@parameter0" + ], + "url": "out/vulnerability/6-ContentProviderPathTraversal.html", + "target": [ + "->@parameter0", + "->$r1", + "->$r2_1", + "->$r5" + ] + }, + "hash": "186d1273a64ac711c703e259ce0329fa8a25cf37", + "possibility": "" +} +``` +[更多关于result格式的解读看这里](startup.md) + +其中`6-ContentProviderPathTraversal.html`中有更加可视化的数据流图. 想要看明白完整的数据流,需要大家对jimple有一定的了解,[查看更多jimple的知识](http://soot-oss.github.io/soot/). + +## case2: getLastPathSegment +发现了漏洞,那么肯定要修复它, 该如何修复呢,这里提供一个修复方式. 通过仔细观察用户传入的路径: +`content://slipme1/?path=../../../../../../../../data/data/com.security.bypasspathtraversal/files/file2` +关键问题是我们的`new File(root, path)`,他实际上做了一个直接的路径拼接,那么只要我不这么做就行了. +一个思路是我希望传入的路径是:`content://slipme2/somefile`,出于安全考虑,**只允许访问sandbox目录下的文件,其子目录下的文件则不可以读取**. +因此,新的设计如下: +```java +public class VulProvider2 extends ContentProvider { + + @Nullable + @Override + public ParcelFileDescriptor openFile(@NonNull Uri uri, @NonNull String mode) throws FileNotFoundException { + File root = getContext().getExternalFilesDir("sandbox"); + String path = uri.getQueryParameter("path"); + return ParcelFileDescriptor.open(new File(root, uri.getLastPathSegment()), ParcelFileDescriptor.MODE_READ_ONLY); + } +} +``` +这时候如果用户传入的uri是: `content://slipme2/../../../../../../../../data/data/com.security.bypasspathtraversal/files/file2`, +那么将只会截取最后的file2作为文件名. 经过发现确实如此. + +### 如何避免误报呢? +如果我们用appshark扫描修复后的代码,会发现依然会报漏洞,这可不是我们希望看到的. 那么怎么避免误报呢? 这就需要sanitizer了. +通过观察修复后的代码发现,这次它是通过`uri.getLastPathSegment()`来获取的路径. 因此可以认为,如果从source传播到sink的路径上, +有`uri.getLastPathSegment()`这样的调用,那么可以认为漏洞已经修复. + +因此添加sanitizer如下: +```json +{ + "sanitize": { + "getLastPathSegment": { + "": { + "TaintCheck": [ + "@this" + ] + } + } + } +} +``` +如果调用了`uri.getLastPathSegment()`,并且this指针被source污染了,那么可以认为漏洞修复了. +被污染了的准确含义是,可能被控制. 比如c=a+b,那么c就被a和b污染了. + +同样按照刚刚的方法重新扫描一下,发现`VulProvider1`的漏洞存在,但是`VulProvider2`的漏洞已经消失了. + + +### 真的修复了? +这里要翻转一下,真的修复了么? +别忘了URL编码问题,如果我们传输的不是`content://slipme2/../../../../../../../../data/data/com.security.bypasspathtraversal/files/file2`, +而是`content://slipme2/encoded/%2E%2E%2F%2E%2E%2F%2E%2E%2F%2E%2E%2F%2E%2E%2F%2E%2E%2F%2E%2E%2F%2E%2E%2Fdata%2Fdata%2Fcom%2Esecurity%2Ebypasspathtraversal%2Ffiles%2Ffile2`,是否可以呢? + +可以验证,VulProvider2也并不是一个有效的修复,仍然有漏洞存在. + + +## case3 检查.. +既然目录遍历漏洞,又称为`Dot dot slash attack`, 说明其核心就是路径中的"../",是关键特征,我们这次直接从这个特征入手,如果path中包含了.., +那么就认为是非法路径即可. + +因此,此次修复方法为: +```java +public class VulProvider3 extends ContentProvider { + + @Nullable + @Override + public ParcelFileDescriptor openFile(@NonNull Uri uri, @NonNull String mode) throws FileNotFoundException { + File root = getContext().getExternalFilesDir("sandbox"); + + String path = uri.getQueryParameter("path"); + File file3 = new File(path); + File internalDir = getContext().getFilesDir(); + try { + if (path.contains("..") || path.startsWith(internalDir.getCanonicalPath())) { + throw new IllegalArgumentException(); + } + } catch (IOException e) { + throw new IllegalArgumentException(); + } + return ParcelFileDescriptor.open(file3, ParcelFileDescriptor.MODE_READ_ONLY); + + } + +} +``` + +注意到这里的条件是两者都不满足: +1. 包含了.. +2. 路径不能以内部路径开头 + +可以很快确认,前两种绕过方式,都已经失效了. + + +### 如何避免误报呢? + +针对这次修复,怎么才能不误报呢? 还是观察这里的限定条件: +1. 包含了.. +2. 路径不能以内部路径开头 + +我们不难想到就是下面的sanitizer: +```json +{ + "sanitize": { + + "containsDotdot": { + "": { + "TaintCheck": [ + "@this" + ], + "p0": [ + "..*" + ] + }, + "": { + "TaintCheck": [ + "@this" + ] + } + } + } +} +``` +那么这个sanitizer的准确含义是什么呢? +针对一条从source到sink的路径上,如果: +1. String.contains的this指针被污染了,并且这个函数调用位置的p0参数能够被"..*"这个常量污染到 +2. 并且String.startWith的this指针也被污染了. + +被污染了的准确含义是,可能被控制. 比如c=a+b,那么c就被a和b污染了. + +同样按照刚刚的方法重新扫描一下,发现`VulProvider1`,`VulProvider2`的漏洞存在,但是`VulProvider3`的漏洞已经消失了. + +### 再次反转,真的修复了么? +你能否想到绕过的方式呢? + + + + +对,那就是软链接,这里有一个明显的问题,就是他校验是如果以app的内部路径开头,就抛出异常. 我们可以通过软链接,既不包含..,也不以app的内部路径开头. +poc代码如下: +```java +String root = getApplicationInfo().dataDir; +String symlink = root + "/symlink"; +android_command("ln -sf /data/data/com.security.bypasspathtraversal/files/file2 " + symlink); +android_command("chmod -R 777 " + root); +String path="content://slipme3/?path=" + symlink; +String data = IOUtils.toString(getContentResolver().openInputStream(Uri.parse(path))); +``` + + +## case4 彻底的修复 +可以存在两种有效的修复方式: + +### 修复方式1 + +```java +public class VulProvider6 extends ContentProvider { + + @Nullable + @Override + public ParcelFileDescriptor openFile(@NonNull Uri uri, @NonNull String mode) throws FileNotFoundException { + File root = getContext().getExternalFilesDir("sandbox"); + + String path = uri.getQueryParameter("path"); + File file3 = new File(path); + try { + if (path.contains("..") || path.startsWith(root.getPath())) { + throw new IllegalArgumentException(); + } + } catch (IOException e) { + throw new IllegalArgumentException(); + } + return ParcelFileDescriptor.open(file3, ParcelFileDescriptor.MODE_READ_ONLY); + + } +} + +``` +注意这里的startsWith 检查的是sandbox的path,所以我们就没法在自己的目录中创建一个软链接了. + +### 修复方式2 + +```java +public class VulProvider5 extends ContentProvider { + + @Nullable + @Override + public ParcelFileDescriptor openFile(@NonNull Uri uri, @NonNull String mode) throws FileNotFoundException { + File root = getContext().getExternalFilesDir("sandbox"); + File file5 = new File(getContext().getExternalFilesDir("sandbox"), uri.getLastPathSegment()); + try { + file5 = file5.getCanonicalFile(); + if (!file5.getPath().startsWith(root.getCanonicalPath())) { + throw new IllegalArgumentException(); + } + } catch (IOException e) { + e.printStackTrace(); + } + return ParcelFileDescriptor.open(file5, ParcelFileDescriptor.MODE_READ_ONLY); + + } +} +``` +这里通过`getCanonicalFile`来解析软链接,这样获取到的就是真实的路径了. 所以这里的条件是: +1. 通过getCanonicalFile获取到真实的路径 +2. 通过startsWith校验真实路径是否以sandbox路径开头. + +这种两种方式都ok,那么如果用了第二种方式,我们怎么避免误报呢? + +不难想到这样的sanitizer: +```json +{ + "getCanonicalFile": { + "": { + "TaintCheck": [ + "@this" + ] + }, + "": { + "TaintCheck": [ + "@this" + ] + } + } +} +``` +这个规则校验的是: +1. getCanonicalFile的this指针要被source污染. +2. startsWith的this指针也要被source污染. + +因此最终的完整规则如下: +```json +{ + "ContentProviderPathTraversal": { + "SliceMode": true, + "traceDepth": 14, + "desc": { + "name": "ContentProviderPathTraversal", + "category": "", + "wiki": "", + "detail": "如果Content Provider重写了openFile，但是没有对Uri进行路径合法性校验，那么攻击者可能通过在uri中插入../的方式访问预期外的文件", + "possibility": "", + "model": "" + }, + "source": { + "Param": { + "<*: android.os.ParcelFileDescriptor openFile(*)>": [ + "p0" + ] + } + }, + "sink": { + "": { + "TaintCheck": [ + "p0" + ] + } + }, + "sanitize": { + "getCanonicalFile": { + "": { + "TaintCheck": [ + "@this" + ] + }, + "": { + "TaintCheck": [ + "@this" + ] + } + }, + "containsDotDot": { + "": { + "TaintCheck": [ + "@this" + ], + "p0": [ + "..*" + ] + }, + "": { + "TaintCheck": [ + "@this" + ] + } + } + } + } +} + + +``` + +## 误报/漏报是无法彻底避免的 + +大家可能有疑问.containsDotDot这个sanitizer存在漏报问题啊,`case3`中的修复方式明明是无效的,但是仍然会被引擎因为是修复了的,这实际上导致了漏报. +这里只能说一下sanitizer的局限性了,它只能根据source污染到的变量的范围来确定要不要去掉一条路径. 真实的修复方式: +`path.startsWith(root.getPath())`和有问题的修复方式` path.startsWith(internalDir.getCanonicalPath())`从形式上看是没什么区别的. +让appshark去识别这种逻辑上的区别,是非常困难的, 这也是appshark的局限. + +## 写在最后 + +appshark是一个实用的基于指针分析的静态分析工具,虽然可以对大型app进行分析,但是不可避免的存在局限性,希望大家能够扬长避短, +在appshark擅长的领域发挥出它的价值, 也为自己的日常工作带来帮助. + +另外,这里有完整的[appshark规则的撰写手册](how_to_write_rules.md) + diff --git a/doc/zh/result.md b/doc/zh/result.md new file mode 100644 index 0000000..2bbdb22 --- /dev/null +++ b/doc/zh/result.md @@ -0,0 +1,84 @@ +# result 解读 + +首先需要说明的是results.json设计的目的就是为了方便程序处理而不是人工阅读. 我们重点关注的是SecurityInfo和ComplianceInfo字段. + +## SecurityInfo + +这里的安全漏洞会根据你在规则中`desc`指定的`category`和`name`进行分类. 方便程序处理,也方便人工阅读. +其中`vulners`字段是这种类型漏洞的列表. 其中每个漏洞都有一个hash字段,该字段可以认为是漏洞的唯一标识. +details字段包含了漏洞的大量信息: + +- Source 规则中source字段匹配到的变量. +- Sink 规则中sink字段匹配到的变量 +- position source对应变量所在的函数 +- entryMethod 分析的入口 +- target 污点在变量之间传播的过程. +- url 以html格式展示的污点在变量之间传播的过程. + +## ComplianceInfo + +ComplianceInfo专门针对隐私合规问题. 如果category是`ComplianceInfo`,那么appshark将会到其特殊处理.比如: + +```json +{ + "desc": { + "name": "GAID_NetworkTransfer_body", + "detail": "存在[获取]相关操作通过网络发送-body", + "category": "ComplianceInfo", + "complianceCategory": "PersonalDeviceInformation_NetworkTransfer", + "complianceCategoryDetail": "个人设备信息_NetworkTransfer", + "level": "3" + } +} +``` + +其分类将是: + +- 第一级是ComplianceInfo +- 第二级是ComplianceCategory指定的PersonalDeviceInformation_NetworkTransfer +- 第三级是name指定的GAID_NetworkTransfer_body. + +比如: + +```json +{ + "ComplianceInfo": { + "PersonalDeviceInformation_NetworkTransfer": { + "GAID_NetworkTransfer_body": { + "category": "ComplianceInfo", + "detail": "存在[获取]相关操作通过网络发送-body", + "name": "GAID_NetworkTransfer_body", + "vulners": [], + "deobfApk": "", + "level": "3" + } + } + } +} +``` + +至于vulners中的字段和SecurityInfo中的含义是一样的. + +## 漏洞详情网页介绍 + +漏洞详情网页设计的目的是,他可以脱离results.json独立展示信息给用户,方便分析漏洞的形成原因. + +### vulnerability detail + +是app的基本信息以及漏洞的基本信息. + +### data flow + +上面的target字段 + +### call stack + +污点传播经历了哪些函数. + +### code detail + +详细展示了污点传播的过程. 如果`config.json5`中指定了javaSource为true,那么还会展示反编译后的函数的java代码. + + + + diff --git a/doc/zh/startup.md b/doc/zh/startup.md new file mode 100644 index 0000000..84d40cc --- /dev/null +++ b/doc/zh/startup.md @@ -0,0 +1,32 @@ +如何快速起步,以扫描一个简单的漏洞为例 + +# 1. 下载jar + +maven仓库提供了完整的jar包,可以下载使用[](). 要求系统安装有jre11环境, + +# 2. 通过github下载config文件夹 + +```txt +git clone +``` + +# 3. 修改config文件 + +1. 将apkPath修改为你想要扫描的apk绝对路径. +2. 指明你要使用的规则,以逗号分隔.并且这些规则应该都放在config/rules目录下. 因为appshark是通过这个路径来查找这些规则的. +3. 指定输出结果保存的目录,默认是当前目录下的out文件,你可以指定一个其他目录. + +# 4. 启动appshark + +```txt +java -jar AppShark-0.1-all.jar config/config.json5 +``` + +# 5. 查看结果 + +结果将在当前目录中的out文件,首先是results.json文件,里面给出了所有的漏洞列表. 关于结果的详细解释请查看[](result.md). +如果对某个具体的漏洞有疑问,可以查看url字段指明的文件. + + + + \ No newline at end of file diff --git a/gradle.properties b/gradle.properties new file mode 100644 index 0000000..29e08e8 --- /dev/null +++ b/gradle.properties @@ -0,0 +1 @@ +kotlin.code.style=official \ No newline at end of file diff --git a/gradle/wrapper/gradle-wrapper.jar b/gradle/wrapper/gradle-wrapper.jar new file mode 100644 index 0000000..e708b1c Binary files /dev/null and b/gradle/wrapper/gradle-wrapper.jar differ diff --git a/gradle/wrapper/gradle-wrapper.properties b/gradle/wrapper/gradle-wrapper.properties new file mode 100644 index 0000000..f371643 --- /dev/null +++ b/gradle/wrapper/gradle-wrapper.properties @@ -0,0 +1,5 @@ +distributionBase=GRADLE_USER_HOME +distributionPath=wrapper/dists +distributionUrl=https\://services.gradle.org/distributions/gradle-7.0-bin.zip +zipStoreBase=GRADLE_USER_HOME +zipStorePath=wrapper/dists diff --git a/gradlew b/gradlew new file mode 100755 index 0000000..4f906e0 --- /dev/null +++ b/gradlew @@ -0,0 +1,185 @@ +#!/usr/bin/env sh + +# +# Copyright 2015 the original author or authors. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# https://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +############################################################################## +## +## Gradle start up script for UN*X +## +############################################################################## + +# Attempt to set APP_HOME +# Resolve links: $0 may be a link +PRG="$0" +# Need this for relative symlinks. +while [ -h "$PRG" ] ; do + ls=`ls -ld "$PRG"` + link=`expr "$ls" : '.*-> \(.*\)$'` + if expr "$link" : '/.*' > /dev/null; then + PRG="$link" + else + PRG=`dirname "$PRG"`"/$link" + fi +done +SAVED="`pwd`" +cd "`dirname \"$PRG\"`/" >/dev/null +APP_HOME="`pwd -P`" +cd "$SAVED" >/dev/null + +APP_NAME="Gradle" +APP_BASE_NAME=`basename "$0"` + +# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"' + +# Use the maximum available, or set MAX_FD != -1 to use that value. +MAX_FD="maximum" + +warn () { + echo "$*" +} + +die () { + echo + echo "$*" + echo + exit 1 +} + +# OS specific support (must be 'true' or 'false'). +cygwin=false +msys=false +darwin=false +nonstop=false +case "`uname`" in + CYGWIN* ) + cygwin=true + ;; + Darwin* ) + darwin=true + ;; + MINGW* ) + msys=true + ;; + NONSTOP* ) + nonstop=true + ;; +esac + +CLASSPATH=$APP_HOME/gradle/wrapper/gradle-wrapper.jar + + +# Determine the Java command to use to start the JVM. +if [ -n "$JAVA_HOME" ] ; then + if [ -x "$JAVA_HOME/jre/sh/java" ] ; then + # IBM's JDK on AIX uses strange locations for the executables + JAVACMD="$JAVA_HOME/jre/sh/java" + else + JAVACMD="$JAVA_HOME/bin/java" + fi + if [ ! -x "$JAVACMD" ] ; then + die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." + fi +else + JAVACMD="java" + which java >/dev/null 2>&1 || die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. + +Please set the JAVA_HOME variable in your environment to match the +location of your Java installation." +fi + +# Increase the maximum file descriptors if we can. +if [ "$cygwin" = "false" -a "$darwin" = "false" -a "$nonstop" = "false" ] ; then + MAX_FD_LIMIT=`ulimit -H -n` + if [ $? -eq 0 ] ; then + if [ "$MAX_FD" = "maximum" -o "$MAX_FD" = "max" ] ; then + MAX_FD="$MAX_FD_LIMIT" + fi + ulimit -n $MAX_FD + if [ $? -ne 0 ] ; then + warn "Could not set maximum file descriptor limit: $MAX_FD" + fi + else + warn "Could not query maximum file descriptor limit: $MAX_FD_LIMIT" + fi +fi + +# For Darwin, add options to specify how the application appears in the dock +if $darwin; then + GRADLE_OPTS="$GRADLE_OPTS \"-Xdock:name=$APP_NAME\" \"-Xdock:icon=$APP_HOME/media/gradle.icns\"" +fi + +# For Cygwin or MSYS, switch paths to Windows format before running java +if [ "$cygwin" = "true" -o "$msys" = "true" ] ; then + APP_HOME=`cygpath --path --mixed "$APP_HOME"` + CLASSPATH=`cygpath --path --mixed "$CLASSPATH"` + + JAVACMD=`cygpath --unix "$JAVACMD"` + + # We build the pattern for arguments to be converted via cygpath + ROOTDIRSRAW=`find -L / -maxdepth 1 -mindepth 1 -type d 2>/dev/null` + SEP="" + for dir in $ROOTDIRSRAW ; do + ROOTDIRS="$ROOTDIRS$SEP$dir" + SEP="|" + done + OURCYGPATTERN="(^($ROOTDIRS))" + # Add a user-defined pattern to the cygpath arguments + if [ "$GRADLE_CYGPATTERN" != "" ] ; then + OURCYGPATTERN="$OURCYGPATTERN|($GRADLE_CYGPATTERN)" + fi + # Now convert the arguments - kludge to limit ourselves to /bin/sh + i=0 + for arg in "$@" ; do + CHECK=`echo "$arg"|egrep -c "$OURCYGPATTERN" -` + CHECK2=`echo "$arg"|egrep -c "^-"` ### Determine if an option + + if [ $CHECK -ne 0 ] && [ $CHECK2 -eq 0 ] ; then ### Added a condition + eval `echo args$i`=`cygpath --path --ignore --mixed "$arg"` + else + eval `echo args$i`="\"$arg\"" + fi + i=`expr $i + 1` + done + case $i in + 0) set -- ;; + 1) set -- "$args0" ;; + 2) set -- "$args0" "$args1" ;; + 3) set -- "$args0" "$args1" "$args2" ;; + 4) set -- "$args0" "$args1" "$args2" "$args3" ;; + 5) set -- "$args0" "$args1" "$args2" "$args3" "$args4" ;; + 6) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" ;; + 7) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" ;; + 8) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" ;; + 9) set -- "$args0" "$args1" "$args2" "$args3" "$args4" "$args5" "$args6" "$args7" "$args8" ;; + esac +fi + +# Escape application args +save () { + for i do printf %s\\n "$i" | sed "s/'/'\\\\''/g;1s/^/'/;\$s/\$/' \\\\/" ; done + echo " " +} +APP_ARGS=`save "$@"` + +# Collect all arguments for the java command, following the shell quoting and substitution rules +eval set -- $DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS "\"-Dorg.gradle.appname=$APP_BASE_NAME\"" -classpath "\"$CLASSPATH\"" org.gradle.wrapper.GradleWrapperMain "$APP_ARGS" + +exec "$JAVACMD" "$@" diff --git a/gradlew.bat b/gradlew.bat new file mode 100644 index 0000000..ac1b06f --- /dev/null +++ b/gradlew.bat @@ -0,0 +1,89 @@ +@rem +@rem Copyright 2015 the original author or authors. +@rem +@rem Licensed under the Apache License, Version 2.0 (the "License"); +@rem you may not use this file except in compliance with the License. +@rem You may obtain a copy of the License at +@rem +@rem https://www.apache.org/licenses/LICENSE-2.0 +@rem +@rem Unless required by applicable law or agreed to in writing, software +@rem distributed under the License is distributed on an "AS IS" BASIS, +@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +@rem See the License for the specific language governing permissions and +@rem limitations under the License. +@rem + +@if "%DEBUG%" == "" @echo off +@rem ########################################################################## +@rem +@rem Gradle startup script for Windows +@rem +@rem ########################################################################## + +@rem Set local scope for the variables with windows NT shell +if "%OS%"=="Windows_NT" setlocal + +set DIRNAME=%~dp0 +if "%DIRNAME%" == "" set DIRNAME=. +set APP_BASE_NAME=%~n0 +set APP_HOME=%DIRNAME% + +@rem Resolve any "." and ".." in APP_HOME to make it shorter. +for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi + +@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script. +set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m" + +@rem Find java.exe +if defined JAVA_HOME goto findJavaFromJavaHome + +set JAVA_EXE=java.exe +%JAVA_EXE% -version >NUL 2>&1 +if "%ERRORLEVEL%" == "0" goto execute + +echo. +echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. +echo. +echo Please set the JAVA_HOME variable in your environment to match the +echo location of your Java installation. + +goto fail + +:findJavaFromJavaHome +set JAVA_HOME=%JAVA_HOME:"=% +set JAVA_EXE=%JAVA_HOME%/bin/java.exe + +if exist "%JAVA_EXE%" goto execute + +echo. +echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% +echo. +echo Please set the JAVA_HOME variable in your environment to match the +echo location of your Java installation. + +goto fail + +:execute +@rem Setup the command line + +set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar + + +@rem Execute Gradle +"%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -classpath "%CLASSPATH%" org.gradle.wrapper.GradleWrapperMain %* + +:end +@rem End local scope for the variables with windows NT shell +if "%ERRORLEVEL%"=="0" goto mainEnd + +:fail +rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of +rem the _cmd.exe /c_ return code! +if not "" == "%GRADLE_EXIT_CONSOLE%" exit 1 +exit /b 1 + +:mainEnd +if "%OS%"=="Windows_NT" endlocal + +:omega diff --git a/run.sh b/run.sh new file mode 100755 index 0000000..cc5ac0c --- /dev/null +++ b/run.sh @@ -0,0 +1,4 @@ +#!/bin/bash +export JAVA_HOME=/usr/local/Cellar/openjdk@11/11.0.12 +export PATH=/usr/local/Cellar/openjdk@11/11.0.12/bin:$PATH +java -jar build/libs/AppShark-0.1-all.jar config/config.json5 \ No newline at end of file diff --git a/settings.gradle.kts b/settings.gradle.kts new file mode 100644 index 0000000..2a553b4 --- /dev/null +++ b/settings.gradle.kts @@ -0,0 +1,5 @@ +/* + * This file was generated by the Gradle 'init' task. + */ + +rootProject.name = "AppShark" \ No newline at end of file diff --git a/src/main/java/net/bytedance/security/app/JavaEntry.java b/src/main/java/net/bytedance/security/app/JavaEntry.java new file mode 100644 index 0000000..f72eb9a --- /dev/null +++ b/src/main/java/net/bytedance/security/app/JavaEntry.java @@ -0,0 +1,8 @@ +package net.bytedance.security.app; + +public class JavaEntry { + + public static void main(String[] args) throws Exception { + KotlinEntry.callMain(args); + } +} diff --git a/src/main/kotlin/META-INF/MANIFEST.MF b/src/main/kotlin/META-INF/MANIFEST.MF new file mode 100644 index 0000000..11a1ca7 --- /dev/null +++ b/src/main/kotlin/META-INF/MANIFEST.MF @@ -0,0 +1,3 @@ +Manifest-Version: 0.1 +Main-Class: net.bytedance.security.app.JavaEntry + diff --git a/src/main/kotlin/at/syntaxerror/json5/DecodeJson5Array.kt b/src/main/kotlin/at/syntaxerror/json5/DecodeJson5Array.kt new file mode 100644 index 0000000..2e47d1a --- /dev/null +++ b/src/main/kotlin/at/syntaxerror/json5/DecodeJson5Array.kt @@ -0,0 +1,63 @@ +/* + * MIT License + * + * Copyright (c) 2021 SyntaxError404 + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ +package at.syntaxerror.json5 + +import kotlinx.serialization.json.JsonArray +import kotlinx.serialization.json.JsonElement + + +/** + * A JSONArray is an array structure capable of holding multiple values, including other JSONArrays + * and [JSONObjects][DecodeJson5Object] + * + * @author SyntaxError404 + */ +class DecodeJson5Array { + + fun decode(parser: JSONParser): JsonArray { + val content: MutableList = mutableListOf() + + if (parser.nextClean() != '[') { + throw parser.createSyntaxException("A JSONArray must begin with '['") + } + while (true) { + var c: Char = parser.nextClean() + when (c) { + Char.MIN_VALUE -> throw parser.createSyntaxException("A JSONArray must end with ']'") + ']' -> break // finish parsing this array + else -> parser.back() + } + val value = parser.nextValue() + content.add(value) + c = parser.nextClean() + when { + c == ']' -> break // finish parsing this array + c != ',' -> throw parser.createSyntaxException("Expected ',' or ']' after value, got '$c' instead") + } + } + + return JsonArray(content) + } + +} diff --git a/src/main/kotlin/at/syntaxerror/json5/DecodeJson5Object.kt b/src/main/kotlin/at/syntaxerror/json5/DecodeJson5Object.kt new file mode 100644 index 0000000..e0e6a47 --- /dev/null +++ b/src/main/kotlin/at/syntaxerror/json5/DecodeJson5Object.kt @@ -0,0 +1,76 @@ +/* + * MIT License + * + * Copyright (c) 2021 SyntaxError404 + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ +package at.syntaxerror.json5 + +import kotlinx.serialization.json.JsonElement +import kotlinx.serialization.json.JsonObject + +/** + * A JSONObject is a map (key-value) structure capable of holding multiple values, including other + * [JSONArrays][DecodeJson5Array] and JSONObjects + * + * @author SyntaxError404 + */ +class DecodeJson5Object( + private val j5: Json5Module, +) { + + fun decode(parser: JSONParser): JsonObject { + + val content: MutableMap = mutableMapOf() + + var c: Char + var key: String + if (parser.nextClean() != '{') { + throw parser.createSyntaxException("A JSONObject must begin with '{'") + } + while (true) { + c = parser.nextClean() + key = when (c) { + Char.MIN_VALUE -> throw parser.createSyntaxException("A JSONObject must end with '}'") + '}' -> break // end of object + else -> { + parser.back() + parser.nextMemberName() + } + } + if (content.containsKey(key)) { + throw JSONException("Duplicate key ${j5.stringify.escapeString(key)}") + } + c = parser.nextClean() + if (c != ':') { + throw parser.createSyntaxException("Expected ':' after a key, got '$c' instead") + } + val value = parser.nextValue() + content[key] = value + c = parser.nextClean() + when { + c == '}' -> break // end of object + c != ',' -> throw parser.createSyntaxException("Expected ',' or '}' after value, got '$c' instead") + } + } + + return JsonObject(content) + } +} diff --git a/src/main/kotlin/at/syntaxerror/json5/JSONException.kt b/src/main/kotlin/at/syntaxerror/json5/JSONException.kt new file mode 100644 index 0000000..a0e3347 --- /dev/null +++ b/src/main/kotlin/at/syntaxerror/json5/JSONException.kt @@ -0,0 +1,42 @@ +/* + * MIT License + * + * Copyright (c) 2021 SyntaxError404 + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ +package at.syntaxerror.json5 + +/** + * An exception used by the JSON5 for Java Library if something went wrong + * + * @author SyntaxError404 + * @version 1.0.0 + */ +open class JSONException( + message: String, + cause: Throwable? = null, +) : RuntimeException(message, cause) { + + class SyntaxError( + message: String, + cause: Throwable? = null, + ) : JSONException(message, cause) + +} diff --git a/src/main/kotlin/at/syntaxerror/json5/JSONOptions.kt b/src/main/kotlin/at/syntaxerror/json5/JSONOptions.kt new file mode 100644 index 0000000..51463db --- /dev/null +++ b/src/main/kotlin/at/syntaxerror/json5/JSONOptions.kt @@ -0,0 +1,68 @@ +/* + * MIT License + * + * Copyright (c) 2021 SyntaxError404 + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ +package at.syntaxerror.json5 + +/** + * This class used is used to customize the behaviour of [parsing][JSONParser] and [stringifying][JSONStringify] + * + * @author SyntaxError404 + * @since 1.1.0 + */ +data class JSONOptions( + /** + * Whether `NaN` should be allowed as a number + * + * Default: `true` + */ + var allowNaN: Boolean = true, + + /** + * Whether `Infinity` should be allowed as a number. + * This applies to both `+Infinity` and `-Infinity` + * + * Default: `true` + */ + var allowInfinity: Boolean = true, + + /** + * Whether invalid unicode surrogate pairs should be allowed + * + * Default: `true` + * + * *This is a [Parser][JSONParser]-only option* + */ + var allowInvalidSurrogates: Boolean = true, + + /** + * Whether string should be single-quoted (`'`) instead of double-quoted (`"`). + * This also includes a [JSONObject's][DecodeJson5Object] member names + * + * Default: `false` + * + * *This is a [Stringify][JSONStringify]-only option* + */ + var quoteSingle: Boolean = false, + + var indentFactor: UInt = 2u +) diff --git a/src/main/kotlin/at/syntaxerror/json5/JSONParser.kt b/src/main/kotlin/at/syntaxerror/json5/JSONParser.kt new file mode 100644 index 0000000..7423fc9 --- /dev/null +++ b/src/main/kotlin/at/syntaxerror/json5/JSONParser.kt @@ -0,0 +1,465 @@ +/* + * MIT License + * + * Copyright (c) 2021 SyntaxError404 + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ +package at.syntaxerror.json5 + +import at.syntaxerror.json5.JSONException.SyntaxError +import kotlinx.serialization.json.JsonElement +import kotlinx.serialization.json.JsonNull +import kotlinx.serialization.json.JsonPrimitive +import java.io.BufferedReader +import java.io.Reader + +/** + * A JSONParser is used to convert a source string into tokens, which then are used to construct + * [JSONObjects][DecodeJson5Object] and [JSONArrays][DecodeJson5Array] + * + * The reader is not [closed][Reader.close] + * + * @author SyntaxError404 + */ +class JSONParser( + reader: Reader, + private val j5: Json5Module, +) { + + private val reader: Reader = if (reader.markSupported()) reader else BufferedReader(reader) + + /** whether the end of the file has been reached */ + private var eof: Boolean = false + + /** whether the current character should be re-read */ + private var back: Boolean = false + + /** the absolute position in the string */ + private var index: Long = -1 + + /** the relative position in the line */ + private var character: Long = 0 + + /** the line number */ + private var line: Long = 1 + + /** the previous character */ + private var previous: Char = Char.MIN_VALUE + + /** the current character */ + private var current: Char = Char.MIN_VALUE + + private val nextCleanToDelimiters: String = ",]}" + + private fun more(): Boolean { + return if (back || eof) { + back && !eof + } else peek().code > 0 + } + + /** Forces the parser to re-read the last character */ + fun back() { + back = true + } + + private fun peek(): Char { + if (eof) { + return Char.MIN_VALUE + } + val c: Int + try { + reader.mark(1) + c = reader.read() + reader.reset() + } catch (e: Exception) { + throw createSyntaxException("Could not peek from source", e) + } + return if (c == -1) Char.MIN_VALUE else c.toChar() + } + + private operator fun next(): Char { + if (back) { + back = false + return current + } + val c: Int = try { + reader.read() + } catch (e: Exception) { + throw createSyntaxException("Could not read from source", e) + } + if (c < 0) { + eof = true + return Char.MIN_VALUE + } + previous = current + current = c.toChar() + index++ + if (isLineTerminator(current) && (current != '\n' || previous != '\r')) { + line++ + character = 0 + } else { + character++ + } + return current + } + + // https://262.ecma-international.org/5.1/#sec-7.3 + private fun isLineTerminator(c: Char): Boolean { + return when (c) { + '\n', '\r', '\u2028', '\u2029' -> true + else -> false + } + } + + // https://spec.json5.org/#white-space + private fun isWhitespace(c: Char): Boolean { + return when (c) { + '\t', '\n', '\u000B', Json5EscapeSequence.FormFeed.char, + '\r', ' ', '\u00A0', '\u2028', '\u2029', '\uFEFF' -> true + else -> + // Unicode category "Zs" (space separators) + Character.getType(c) == Character.SPACE_SEPARATOR.toInt() + } + } + + private fun nextMultiLineComment() { + while (true) { + val n = next() + if (n == '*' && peek() == '/') { + next() + return + } + } + } + + private fun nextSingleLineComment() { + while (true) { + val n = next() + if (isLineTerminator(n) || n.code == 0) { + return + } + } + } + + /** + * Reads until encountering a character that is not a whitespace according to the + * [JSON5 Specification](https://spec.json5.org/#white-space) + * + * @return a non-whitespace character, or `0` if the end of the stream has been reached + */ + fun nextClean(): Char { + while (true) { + if (!more()) { + throw createSyntaxException("Unexpected end of data") + } + val n = next() + if (n == '/') { + when (peek()) { + '*' -> { + next() + nextMultiLineComment() + } + '/' -> { + next() + nextSingleLineComment() + } + else -> { + return n + } + } + } else if (!isWhitespace(n)) { + return n + } + } + } + + private fun nextCleanTo(delimiters: String = nextCleanToDelimiters): String { + val result = StringBuilder() + while (true) { + if (!more()) { + throw createSyntaxException("Unexpected end of data") + } + val n = nextClean() + if (delimiters.indexOf(n) > -1 || isWhitespace(n)) { + back() + break + } + result.append(n) + } + return result.toString() + } + + private fun deHex(c: Char): Int? { + return when (c) { + in '0'..'9' -> c - '0' + in 'a'..'f' -> c - 'a' + 0xA + in 'A'..'F' -> c - 'A' + 0xA + else -> null + } + } + + private fun unicodeEscape(member: Boolean, part: Boolean): Char { + var value = "" + var codepoint = 0 + for (i in 0..3) { + val n = next() + value += n + val hex = deHex(n) + ?: throw createSyntaxException("Illegal unicode escape sequence '\\u$value' in ${if (member) "key" else "string"}") + codepoint = codepoint or (hex shl (3 - i shl 2)) + } + if (member && !isMemberNameChar(codepoint.toChar(), part)) { + throw createSyntaxException("Illegal unicode escape sequence '\\u$value' in key") + } + return codepoint.toChar() + } + + private fun checkSurrogate(hi: Char, lo: Char) { + if (j5.options.allowInvalidSurrogates) { + return + } + if (!Character.isHighSurrogate(hi) || !Character.isLowSurrogate(lo)) { + return + } + if (!Character.isSurrogatePair(hi, lo)) { + throw createSyntaxException( + String.format( + "Invalid surrogate pair: U+%04X and U+%04X", + hi, lo + ) + ) + } + } + + // https://spec.json5.org/#prod-JSON5String + private fun nextString(quote: Char): String { + val result = StringBuilder() + var value: String + var codepoint: Int + var n = 0.toChar() + var prev: Char + while (true) { + if (!more()) { + throw createSyntaxException("Unexpected end of data") + } + prev = n + n = next() + if (n == quote) { + break + } + if (isLineTerminator(n) && n.code != 0x2028 && n.code != 0x2029) { + throw createSyntaxException("Unescaped line terminator in string") + } + if (n == '\\') { + n = next() + if (isLineTerminator(n)) { + if (n == '\r' && peek() == '\n') { + next() + } + // escaped line terminator/ line continuation + continue + } else { + when (n) { + '\'', '"', '\\' -> { + result.append(n) + continue + } + 'b' -> { + result.append('\b') + continue + } + 'f' -> { + result.append(Json5EscapeSequence.FormFeed.char) + continue + } + 'n' -> { + result.append('\n') + continue + } + 'r' -> { + result.append('\r') + continue + } + 't' -> { + result.append('\t') + continue + } + 'v' -> { + result.append(0x0B.toChar()) + continue + } + '0' -> { + val p = peek() + if (p.isDigit()) { + throw createSyntaxException("Illegal escape sequence '\\0$p'") + } + result.append(0.toChar()) + continue + } + 'x' -> { + value = "" + codepoint = 0 + var i = 0 + while (i < 2) { + n = next() + value += n + val hex = deHex(n) + ?: throw createSyntaxException("Illegal hex escape sequence '\\x$value' in string") + codepoint = codepoint or (hex shl (1 - i shl 2)) + ++i + } + n = codepoint.toChar() + } + 'u' -> n = unicodeEscape(member = false, part = false) + else -> if (n.isDigit()) { + throw SyntaxError("Illegal escape sequence '\\$n'") + } + } + } + } + checkSurrogate(prev, n) + result.append(n) + } + return result.toString() + } + + private fun isMemberNameChar(n: Char, isNotEmpty: Boolean): Boolean { + if (n == '$' || n == '_' || n.code == 0x200C || n.code == 0x200D) { + return true + } + + return when (n.category) { + + CharCategory.UPPERCASE_LETTER, + CharCategory.LOWERCASE_LETTER, + CharCategory.TITLECASE_LETTER, + CharCategory.MODIFIER_LETTER, + CharCategory.OTHER_LETTER, + CharCategory.LETTER_NUMBER -> return true + + CharCategory.NON_SPACING_MARK, + CharCategory.COMBINING_SPACING_MARK, + CharCategory.DECIMAL_DIGIT_NUMBER, + CharCategory.CONNECTOR_PUNCTUATION -> isNotEmpty + + else -> return false + } + } + + /** + * Reads a member name from the source according to the + * [JSON5 Specification](https://spec.json5.org/#prod-JSON5MemberName) + */ + fun nextMemberName(): String { + val result = StringBuilder() + var prev: Char + var n = next() + if (n == '"' || n == '\'') { + return nextString(n) + } + back() + n = 0.toChar() + while (true) { + if (!more()) { + throw createSyntaxException("Unexpected end of data") + } + val isNotEmpty = result.isNotEmpty() + prev = n + n = next() + if (n == '\\') { // unicode escape sequence + n = next() + if (n != 'u') { + throw createSyntaxException("Illegal escape sequence '\\$n' in key") + } + n = unicodeEscape(true, isNotEmpty) + } else if (!isMemberNameChar(n, isNotEmpty)) { + back() + break + } + checkSurrogate(prev, n) + result.append(n) + } + if (result.isEmpty()) { + throw createSyntaxException("Empty key") + } + return result.toString() + } + + /** + * Reads a value from the source according to the + * [JSON5 Specification](https://spec.json5.org/#prod-JSON5Value) + */ + fun nextValue(): JsonElement { + when (val n = nextClean()) { + '"', '\'' -> { + val string = nextString(n) + return JsonPrimitive(string) + } + '{' -> { + back() + return j5.objectDecoder.decode(this) + } + '[' -> { + back() + return j5.arrayDecoder.decode(this) + } + } + back() + val string = nextCleanTo() + return when { + string == "null" -> JsonNull + PATTERN_BOOLEAN.matches(string) -> JsonPrimitive(string == "true") + + // val bigint = BigInteger(string) + // return bigint + PATTERN_NUMBER_INTEGER.matches(string) -> JsonPrimitive(string.toLong()) + + PATTERN_NUMBER_FLOAT.matches(string) + || PATTERN_NUMBER_NON_FINITE.matches(string) -> { + try { + JsonPrimitive(string.toDouble()) + } catch (e: NumberFormatException) { + throw createSyntaxException("could not parse number '$string'") + } + } + PATTERN_NUMBER_HEX.matches(string) -> { + val hex = string.uppercase().split("0X").joinToString("") + JsonPrimitive(hex.toLong(16)) + } + else -> throw JSONException("Illegal value '$string'") + } + } + + fun createSyntaxException(message: String, cause: Throwable? = null): SyntaxError = + SyntaxError("$message, at index $index, character $character, line $line]", cause) + + companion object { + private val PATTERN_BOOLEAN = + Regex("true|false") + private val PATTERN_NUMBER_FLOAT = + Regex("[+-]?((0|[1-9]\\d*)(\\.\\d*)?|\\.\\d+)([eE][+-]?\\d+)?") + private val PATTERN_NUMBER_INTEGER = + Regex("[+-]?(0|[1-9]\\d*)") + private val PATTERN_NUMBER_HEX = + Regex("[+-]?0[xX][0-9a-fA-F]+") + private val PATTERN_NUMBER_NON_FINITE = + Regex("[+-]?(Infinity|NaN)") + } +} diff --git a/src/main/kotlin/at/syntaxerror/json5/JSONStringify.kt b/src/main/kotlin/at/syntaxerror/json5/JSONStringify.kt new file mode 100644 index 0000000..e1a5d97 --- /dev/null +++ b/src/main/kotlin/at/syntaxerror/json5/JSONStringify.kt @@ -0,0 +1,192 @@ +/* + * MIT License + * + * Copyright (c) 2021 SyntaxError404 + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in all + * copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE + * SOFTWARE. + */ +package at.syntaxerror.json5 + +import kotlinx.serialization.json.JsonArray +import kotlinx.serialization.json.JsonObject + +/** + * A utility class for serializing [JSONObjects][DecodeJson5Object] and [JSONArrays][DecodeJson5Array] + * into their string representations + * + * @author SyntaxError404 + */ +class JSONStringify( + private val options: JSONOptions +) { + + private val quoteToken = if (options.quoteSingle) '\'' else '"' + private val emptyString = "$quoteToken$quoteToken" + private val indentFactor = options.indentFactor + + /** + * Converts a JSONObject into its string representation. The indentation factor enables + * pretty-printing and defines how many spaces (' ') should be placed before each key/value pair. + * A factor of `< 1` disables pretty-printing and discards any optional whitespace + * characters. + * + * `indentFactor = 2`: + * ``` + * { + * "key0": "value0", + * "key1": { + * "nested": 123 + * }, + * "key2": false + * } + * ``` + * + * `indentFactor = 0`: + * + * ``` + * {"key0":"value0","key1":{"nested":123},"key2":false} + * ``` + */ + fun encodeObject( + jsonObject: JsonObject, + indent: String = "", + ): String { + val childIndent = indent + " ".repeat(indentFactor.toInt()) + val isIndented = indentFactor > 0u + + val sb = StringBuilder() + sb.append('{') + jsonObject.forEach { (key, value) -> + if (sb.length != 1) { + sb.append(',') + } + if (isIndented) { + sb.append('\n').append(childIndent) + } + sb.append(escapeString(key)).append(':') + if (isIndented) { + sb.append(' ') + } + sb.append(encode(value, childIndent)) + } + if (isIndented) { + sb.append('\n').append(indent) + } + sb.append('}') + return sb.toString() + } + + /** + * Converts a JSONArray into its string representation. The indentation factor enables + * pretty-printing and defines how many spaces (' ') should be placed before each value. A factor + * of `< 1` disables pretty-printing and discards any optional whitespace characters. + * + * + * `indentFactor = 2`: + * ``` + * [ + * "value", + * { + * "nested": 123 + * }, + * false + * ] + * ``` + * + * `indentFactor = 0`: + * ``` + * ["value",{"nested":123},false] + * ``` + */ + fun encodeArray( + array: JsonArray, + indent: String = "", + ): String { + val childIndent = indent + " ".repeat(indentFactor.toInt()) + val isIndented = indentFactor > 0u + + val sb = StringBuilder() + sb.append('[') + for (value in array) { + if (sb.length != 1) { + sb.append(',') + } + if (isIndented) { + sb.append('\n').append(childIndent) + } + sb.append(encode(value, childIndent)) + } + if (isIndented) { + sb.append('\n').append(indent) + } + sb.append(']') + return sb.toString() + } + + private fun encode( + value: Any?, + indent: String, + ): String { + return when (value) { + null -> "null" + is JsonObject -> encodeObject(value, indent) + is JsonArray -> encodeArray(value, indent) + is String -> escapeString(value) + is Double -> { + when { + !options.allowNaN && value.isNaN() -> throw JSONException("Illegal NaN in JSON") + !options.allowInfinity && value.isInfinite() -> throw JSONException("Illegal Infinity in JSON") + else -> value.toString() + } + } + else -> value.toString() + } + } + + fun escapeString(string: String?): String { + return if (string.isNullOrEmpty()) { + emptyString + } else { + string + .asSequence() + .joinToString( + separator = "", + prefix = quoteToken.toString(), + postfix = quoteToken.toString() + ) { c: Char -> + + val formattedChar: String? = when (c) { + quoteToken -> "\\$quoteToken" + in Json5EscapeSequence.escapableChars -> Json5EscapeSequence.asEscapedString(c) + else -> when (c.category) { + CharCategory.FORMAT, + CharCategory.LINE_SEPARATOR, + CharCategory.PARAGRAPH_SEPARATOR, + CharCategory.CONTROL, + CharCategory.PRIVATE_USE, + CharCategory.SURROGATE, + CharCategory.UNASSIGNED -> String.format("\\u%04X", c) + else -> null + } + } + formattedChar ?: c.toString() + } + } + } +} diff --git a/src/main/kotlin/at/syntaxerror/json5/Json5EscapeSequence.kt b/src/main/kotlin/at/syntaxerror/json5/Json5EscapeSequence.kt new file mode 100644 index 0000000..b4d9c1a --- /dev/null +++ b/src/main/kotlin/at/syntaxerror/json5/Json5EscapeSequence.kt @@ -0,0 +1,31 @@ +package at.syntaxerror.json5 + +/** https://spec.json5.org/#escapes */ +enum class Json5EscapeSequence( + val char: Char, + val escaped: String, +) { + //@formatter:off + Apostrophe('\u0027', "\\'"), + QuotationMark('\u0022', "\\\""), + ReverseSolidus('\u005C', "\\\\"), + Backspace('\u0008', "\\b"), + FormFeed('\u000C', "\\f"), + LineFeed('\u000A', "\\n"), + CarriageReturn('\u000D', "\\r"), + HorizontalTab('\u0009', "\\t"), + VerticalTab('\u000B', "\\v"), + Null('\u0000', "\\0"), + //@formatter:on + ; + + companion object { + private val mapCharToRepresentation = values().associate { it.char to it.escaped } + + val escapableChars = values().map { it.char } + + fun asEscapedString(char: Char): String? = mapCharToRepresentation[char] + + fun isEscapable(char: Char) = mapCharToRepresentation.containsKey(char) + } +} diff --git a/src/main/kotlin/at/syntaxerror/json5/Json5Module.kt b/src/main/kotlin/at/syntaxerror/json5/Json5Module.kt new file mode 100644 index 0000000..e5cd815 --- /dev/null +++ b/src/main/kotlin/at/syntaxerror/json5/Json5Module.kt @@ -0,0 +1,46 @@ +package at.syntaxerror.json5 + + +import kotlinx.serialization.json.JsonArray +import kotlinx.serialization.json.JsonObject +import java.io.InputStream +import java.io.InputStreamReader +import java.io.Reader + +class Json5Module( + configure: JSONOptions.() -> Unit = {} +) { + internal val options: JSONOptions = JSONOptions() + internal val stringify: JSONStringify = JSONStringify(options) + + internal val arrayDecoder = DecodeJson5Array() + internal val objectDecoder = DecodeJson5Object(this) + + init { + options.configure() + } + + fun decodeObject(string: String): JsonObject = decodeObject(string.reader()) + fun decodeObject(stream: InputStream): JsonObject = decodeObject(InputStreamReader(stream)) + + fun decodeObject(reader: Reader): JsonObject { + return reader.use { r -> + val parser = JSONParser(r, this) + objectDecoder.decode(parser) + } + } + + fun decodeArray(string: String): JsonArray = decodeArray(string.reader()) + fun decodeArray(stream: InputStream): JsonArray = decodeArray(InputStreamReader(stream)) + + fun decodeArray(reader: Reader): JsonArray { + return reader.use { r -> + val parser = JSONParser(r, this) + arrayDecoder.decode(parser) + } + } + + fun encodeToString(array: JsonArray) = stringify.encodeArray(array) + fun encodeToString(jsonObject: JsonObject) = stringify.encodeObject(jsonObject) + +} diff --git a/src/main/kotlin/at/syntaxerror/json5/LICENSE b/src/main/kotlin/at/syntaxerror/json5/LICENSE new file mode 100644 index 0000000..0193993 --- /dev/null +++ b/src/main/kotlin/at/syntaxerror/json5/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2021 SyntaxError404 + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/src/main/kotlin/at/syntaxerror/json5/README.md b/src/main/kotlin/at/syntaxerror/json5/README.md new file mode 100644 index 0000000..da10cad --- /dev/null +++ b/src/main/kotlin/at/syntaxerror/json5/README.md @@ -0,0 +1,117 @@ +# WORK IN PROGRESS + +I'm hacking around with this at the moment - almost nothing is tested. + +The intention is to integrate with Kotlinx Serialization. + +# json5 Kotlin + +A JSON5 Library for Kotlin 1.6, Java 11 + +## Overview + +The [JSON5 Standard](https://json5.org/) tries to make JSON more human-readable + +This is a reference implementation, capable of parsing JSON5 data according to +the [specification](https://spec.json5.org/). + +## Getting started + +Gradle (Kotlin): + +```kotlin +repositories { + maven("https://jitpack.io") +} + +dependencies { + implementation("adamko-dev:json5-kotlin:${version}") +} +``` + +Maven: + +```xml + + + jitpack.io + https://jitpack.io + + +``` + +```xml + + + adamko-dev + json5-kotlin + ${json5-kotlin.version} + + +``` + +### Usage + +```kotlin +import at.syntaxerror.json5.Json5Module +import kotlinx.serialization.json.JsonObject + +// create and configure the Json5Module +val j5 = Json5Module { + allowInfinity = true + indentFactor = 4u +} + +val json5 = """ + { + // comments + unquoted: 'and you can quote me on that', + singleQuotes: 'I can use "double quotes" here', + lineBreaks: "Look, Mom! \ + No \\n's!", + hexadecimal: 0xdecaf, + leadingDecimalPoint: .8675309, + andTrailing: 8675309., + positiveSign: +1, + trailingComma: 'in objects', + andIn: [ + 'arrays', + ], + "backwardsCompatible": "with JSON", + } + """.trimIndent() + +// Parse a JSON5 String to a Kotlinx Serialization JsonObject +val jsonObject: JsonObject = j5.decodeObject(json5) + +// encode the JsonObject to a Json5 String +val jsonString = j5.encodeToString(jsonObject) + +println(jsonString) +/* +{ + "unquoted": "and you can quote me on that", + "singleQuotes": "I can use \"double quotes\" here", + "lineBreaks": "Look, Mom! No \\n's!", + "hexadecimal": 912559, + "leadingDecimalPoint": 0.8675309, + "andTrailing": 8675309.0, + "positiveSign": 1, + "trailingComma": "in objects", + "andIn": [ + "arrays" + ], + "backwardsCompatible": "with JSON" +} +*/ +``` + +## Credits + +This project is entirely based on [Synt4xErr0r4/json5](https://github.com/Synt4xErr0r4/json5/), +which was partly based on stleary's [JSON-Java](https://github.com/stleary/JSON-java) library. + +## License + +This project is licensed under +the [MIT License](https://github.com/Synt4xErr0r4/json5/blob/main/LICENSE) diff --git a/src/main/kotlin/net/bytedance/security/app/AnalyzeStepByStep.kt b/src/main/kotlin/net/bytedance/security/app/AnalyzeStepByStep.kt new file mode 100644 index 0000000..3a0978a --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/AnalyzeStepByStep.kt @@ -0,0 +1,195 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app + +import kotlinx.coroutines.* +import net.bytedance.security.app.Log.logInfo +import net.bytedance.security.app.result.OutputSecResults +import net.bytedance.security.app.ruleprocessor.RuleProcessorFactory +import net.bytedance.security.app.ruleprocessor.TaintFlowRuleProcessor +import net.bytedance.security.app.rules.RuleFactory +import net.bytedance.security.app.rules.Rules +import net.bytedance.security.app.taintflow.TaintAnalyzer +import net.bytedance.security.app.util.profiler +import soot.Scene +import soot.SootClass +import soot.options.Options +import java.io.File + +class AnalyzeStepByStep { + suspend fun loadRules( + ruleList: List, + ): Rules { + val rulePathList = ruleList.map { + "${getConfig().rulePath}/$it" + }.toList() + val rules = Rules(rulePathList, RuleFactory()) + rules.loadRules() + return rules + } + + suspend fun parseRules(ctx: PreAnalyzeContext, rules: Rules): List { + val jobs = ArrayList() + val analyzers = ArrayList() + val scope = CoroutineScope(Dispatchers.Default) + for (r in rules.allRules) { + val rp = RuleProcessorFactory.create(ctx, r.mode) + val job = scope.launch(CoroutineName("parseRules-${r.name}")) { + rp.process(r) + if (rp is TaintFlowRuleProcessor) { + if (analyzers.size > getConfig().ruleMaxAnalyzer) { + logInfo("rule ${r.name} has too many rules: ${analyzers.size}, dropped") + return@launch + } + synchronized(analyzers) { + analyzers.addAll(rp.analyzers) + } + + } + } + jobs.add(job) + } + + jobs.joinAll() + logInfo("analyzers: ${analyzers.size}") + profiler.setAnalyzers(analyzers) + ctx.callGraph.clear() + if (false) { + PLUtils.dumpClass(PLUtils.CUSTOM_CLASS) + var first = true + for (a in analyzers) { + if (first) { + println(a.dump()) + first = false + } + println("entry:${a.entryMethod.signature}") + } + } + return analyzers + } + + suspend fun createContext(rules: Rules): PreAnalyzeContext { + val preAnalyzeContext = PreAnalyzeContext() + preAnalyzeContext.createContext(rules, getConfig().callBackEnhance) + return preAnalyzeContext + } + + enum class TYPE { + CLASS, APK, AAR, JIMPLE + } + + fun setExclude() { + // reduce time + val excludeList = ArrayList() + excludeList.add("java.*") + excludeList.add("org.*") + excludeList.add("sun.*") + // excludeList.add("android.*"); +// excludeList.add("androidx.*"); + Options.v().set_exclude(excludeList) + // do not load body in exclude list + Options.v().set_no_bodies_for_excluded(true) + Scene.v().addBasicClass("android.os.Handler") + Scene.v().addBasicClass("java.lang.Object[]", SootClass.HIERARCHY) + Scene.v().addBasicClass("java.beans.Transient", SootClass.SIGNATURES) + Scene.v().addBasicClass("java.time.ZoneRegion", SootClass.SIGNATURES) + } + + fun initSoot( + type: TYPE, + targetPath: String, + sdkPath: String, + outPath: String + ) { + Log.logDebug("Init soot for $targetPath") + + if (type == TYPE.CLASS) { + Options.v().set_src_prec(Options.src_prec_class) + Options.v().set_process_dir(listOf(targetPath)) + } else if (type == TYPE.JIMPLE) { + Options.v().set_src_prec(Options.src_prec_jimple) + Options.v().set_process_dir(listOf(targetPath)) + } else if (type == TYPE.APK) { + Options.v().set_src_prec(Options.src_prec_apk) + + // get android.jar path by sdk path and API level of apk + val androidJarPath = Scene.v().getAndroidJarPath(sdkPath, targetPath) + val processPathList: MutableList = ArrayList() + processPathList.add(targetPath) + processPathList.add(androidJarPath) + val platformFile = File(sdkPath) + if (platformFile.exists() && platformFile.isDirectory) { + for (jarFile in platformFile.listFiles()!!) { + if (jarFile.extension != "jar") { + continue //exclude non-jar file + } + processPathList.add(jarFile.absolutePath) + } + } + Options.v().set_process_dir(processPathList) + Options.v().set_force_android_jar(androidJarPath) + Options.v().set_process_multiple_dex(true) + } else if (type == TYPE.AAR) { + Options.v().set_src_prec(Options.src_prec_class) + Options.v().set_process_dir(listOf(targetPath)) + } + // set the output dir + Options.v().set_output_dir(outPath) + + // output jimple + Options.v().set_output_format(Options.output_format_jimple) + Options.v().set_allow_phantom_refs(true) + Options.v().set_whole_program(true) + Options.v().set_keep_line_number(false) + Options.v().set_wrong_staticness(Options.wrong_staticness_ignore) + Options.v().set_debug(false) + Options.v().set_verbose(false) + Options.v().set_validate(false) +// Options.v().set_keep_line_number(true) + setExclude() + logInfo("loadNecessaryClasses") + try { + Scene.v().loadNecessaryClasses() // may take dozens of seconds + } catch (ex: Exception) { + Log.logErr("loadNecessaryClasses error: ${ex.message}") + throw ex + } + logInfo("loadNecessaryClasses Done classes=${Scene.v().classes.size}") + } + + + suspend fun solve(ctx: PreAnalyzeContext, analyzers: List) { + if (getConfig().doWholeProcessMode) { + solveWholeProcess(ctx, analyzers) + } else { + solveSliceMode(ctx, analyzers) + } + //generate report + OutputSecResults.processResult(ctx) + } + + private suspend fun solveWholeProcess(ctx: PreAnalyzeContext, analyzers: List) { + val p = WholeProcessAnalyzeWrapper(ctx, analyzers) + p.run() + } + + private suspend fun solveSliceMode(ctx: PreAnalyzeContext, analyzers: List) { + val p = SliceAnalyzeWrapper(ctx, analyzers) + p.run() + } +} diff --git a/src/main/kotlin/net/bytedance/security/app/ArgumentConfig.kt b/src/main/kotlin/net/bytedance/security/app/ArgumentConfig.kt new file mode 100644 index 0000000..503d41a --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/ArgumentConfig.kt @@ -0,0 +1,104 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app + +import kotlinx.serialization.SerialName +import kotlinx.serialization.Serializable + +@Serializable +class ArgumentConfig( + @SerialName("CallBackEnhance") + var callBackEnhance: Boolean = false, + @SerialName("ManifestTrace") + var manifestTrace: Int = 3, + var apkPath: String, + var configPath: String = "", + //max pointer analysis time in second for each entry point + var maxPointerAnalyzeTime: Int = 600, + var javaSource: Boolean? = false, + /** + * If you have OOM problems, try lowering this value, such as 1, to save memory + */ + var maxThread: Int = 2, // Runtime.getRuntime().availableProcessors(), + @SerialName("out") var outPath: String = "out", + var rulePath: String = "", + var rules: String = "", + var supportFragment: Boolean = false, + var logLevel: Int = INFO, + var ruleMaxAnalyzer: Int = 5000, // If a rule produces too many Analyzers, the rule is discarded + var deobfApk: String = "", // Decompiled APK download address + var debugRule: String = "", + /** + * If you want accurate results, it's best to have full-program analyze mode, + * but it doesn't support large apps because it's too slow + */ + var doWholeProcessMode: Boolean = false, + var maxPathLength: Int = 32, // max taint flow path + var skipAnalyzeNonRelatedMethods: Boolean = false, // skip analyze non-related methods with source and sinks ,if skip may lead to false negatives. + var skipPointerPropagationForLibraryMethod: Boolean = true, //skip pointer propagation for library methods,if skip may lead to false negatives. + //if exists, use it to replace Package in EngineConfig.json5 + var libraryPackage: List? = null, +) { + companion object { + val defaultConfig: ArgumentConfig + + init { + val wd = System.getProperty("user.dir") + defaultConfig = ArgumentConfig( + apkPath = "$wd/app.apk", + outPath = "$wd/out", + rules = "", + javaSource = false, + maxPointerAnalyzeTime = 600, + maxThread = 4, + manifestTrace = 3, + callBackEnhance = true, + doWholeProcessMode = false, + deobfApk = "$wd/app.apk", + logLevel = 1, + configPath = "$wd/config", + rulePath = "$wd/config/rules", + ) + } + + fun mergeWithDefaultConfig(cfg: ArgumentConfig) { + if (cfg.outPath.isEmpty()) { + cfg.outPath = defaultConfig.outPath + } + if (cfg.configPath.isEmpty()) { + cfg.configPath = defaultConfig.configPath + } + if (cfg.rulePath.isEmpty()) { + cfg.rulePath = "${cfg.configPath}/rules" + } + } + } +} + +var cfg: ArgumentConfig? = null + +/** + * ArgumentConfig is a global variable,it must be set as early as possible + */ +fun getConfig(): ArgumentConfig { + if (cfg != null) { + return cfg as ArgumentConfig + } + cfg = ArgumentConfig.defaultConfig + return cfg!! +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/EngineInfo.kt b/src/main/kotlin/net/bytedance/security/app/EngineInfo.kt new file mode 100644 index 0000000..9961899 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/EngineInfo.kt @@ -0,0 +1,24 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app + + +object EngineInfo { + //version info + const val Version = "0.1" +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/Fragment.kt b/src/main/kotlin/net/bytedance/security/app/Fragment.kt new file mode 100644 index 0000000..87be278 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/Fragment.kt @@ -0,0 +1,325 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app + +import net.bytedance.security.app.android.AndroidUtils +import net.bytedance.security.app.android.AndroidUtils.findFragmentsInLayout +import net.bytedance.security.app.android.LifecycleConst +import net.bytedance.security.app.preprocess.Patch.resolveMethodException +import soot.Scene +import soot.SootClass +import soot.SootMethod +import soot.Value +import soot.jimple.* +import soot.jimple.internal.JAssignStmt +import soot.jimple.internal.JCastExpr +import soot.jimple.internal.JimpleLocal + +/** +1. Find the Fragment in the Layout file and associate it with the Activity +2. Find the Fragment that is dynamically added in the code, generate the calling code, and inject it into the related Activity code. + */ +class Fragment(val ctx: PreAnalyzeContext) { + private val androidFragmentClassName = "androidx.fragment.app.Fragment" + + /* + key: subclass of androidx.fragment.app.Fragment + value: callback method + */ + var fragmentClassName2GeneratedEntry: MutableMap = HashMap() + + var reachableDepth = 10 + fun findFragmentAndAddCallbackToEntry(entry: SootMethod) { + fragmentClassName2GeneratedEntry.clear() + val foundFragmentClass: MutableSet = HashSet() + val layoutFragments = findFragmentByLayoutFile(entry) + if (layoutFragments != null) { + foundFragmentClass.addAll(layoutFragments) + } + lookupReferencedFragmentClassForOneEntry(entry, foundFragmentClass) + callFragmentEntryMethod(entry) + } + + private fun callFragmentEntryMethod(entry: SootMethod) { + val units = entry.activeBody.units + units.removeLast() + for (m in fragmentClassName2GeneratedEntry.values) { + val argType = m.getParameterType(0) + val args: MutableList = ArrayList() + val localArg = Jimple.v().newLocal("v0", argType) + args.add(localArg) + val invokeStmt = Jimple.v().newInvokeStmt( + Jimple.v().newStaticInvokeExpr( + m.makeRef(), args + ) + ) + units.add(invokeStmt) + } + // insert "return" + units.add(Jimple.v().newReturnVoidStmt()) + } + + + private fun lookupReferencedFragmentClassForOneEntry( + entry: SootMethod, + foundFragmentClass: MutableSet + ): Set { + val visited: MutableSet = HashSet() + queryFragmentByEntryRecursive(entry, reachableDepth, visited, foundFragmentClass) + if (foundFragmentClass.isNotEmpty()) { + addFragmentDynamic(foundFragmentClass, HashSet(foundFragmentClass), visited) + } + return foundFragmentClass + } + + private fun setSootClass2SetString(s: Set): MutableSet { + val r: MutableSet = HashSet() + for (clz in s) { + r.add(clz.name) + } + return r + } + + + private fun addNewFoundFragmentClassesToTotal( + totalFragmentSet: MutableSet, + newClasses: Set + ): Set { + val filtered: MutableSet = HashSet() + if (newClasses.isEmpty()) { + return filtered + } + val totalStrings: Set = setSootClass2SetString(totalFragmentSet) + val newStrings = setSootClass2SetString(newClasses) + newStrings.removeAll(totalStrings) + if (newStrings.isNotEmpty()) { + for (clz in newClasses) { + for (s in newStrings) { + if (clz.name == s) { + totalFragmentSet.add(clz) + filtered.add(clz) + } + } + } + } + return filtered + } + + + private fun addFragmentDynamic( + totalFragmentSet: MutableSet, + currentFragmentSet: Set, + visited: MutableSet + ) { + val filteredNewClasses: MutableSet = HashSet() + for (fragmentClass in currentFragmentSet) { + val fragmentEntry = + PLUtils.createComponentEntry(androidFragmentClass, fragmentClass, LifecycleConst.FragmentMethods) + fragmentClassName2GeneratedEntry[fragmentClass.name] = fragmentEntry + + val newClasses: MutableSet = HashSet() + queryFragmentByEntryRecursive(fragmentEntry, reachableDepth, visited, newClasses) + filteredNewClasses.addAll(addNewFoundFragmentClassesToTotal(totalFragmentSet, newClasses)) + } + if (filteredNewClasses.isNotEmpty()) { + addFragmentDynamic(totalFragmentSet, filteredNewClasses, visited) + } + } + + private fun queryFragmentByEntryRecursive( + entry: SootMethod, + depth: Int, + visited: MutableSet, + visitedFragmentClass: MutableSet + ) { + if (depth <= 0) { + return + } + if (visited.contains(entry)) { + return + } + visited.add(entry) + val callees = ctx.callGraph.heirCallGraph[entry] ?: return + val methods = HashSet(fragmentTransactionMethods.keys) + methods.retainAll(callees) + if (methods.isNotEmpty()) { + for (unit in entry.activeBody.units) { + val stmt = unit as Stmt + if (stmt.containsInvokeExpr()) { + val invokeExpr = stmt.invokeExpr + var value: Value? + val i = fragmentTransactionMethods[resolveMethodException(invokeExpr)] ?: continue + value = invokeExpr.args[i] + val clz = Scene.v().getSootClassUnsafe(value.type.toString(), false) + if (androidFragmentClassName == clz.name) { + val s = findOneStepFragment(entry, value as JimpleLocal) + visitedFragmentClass.addAll(s) + continue + } + addFragmentSootClassToSet(visitedFragmentClass, clz) + } + } + } + for (methodSignature in callees) { + queryFragmentByEntryRecursive(methodSignature, depth - 1, visited, visitedFragmentClass) + } + } + + + private fun findOneStepFragment(m: SootMethod, fragmentLocal: JimpleLocal): Set { + val classes: MutableSet = HashSet() + for (unit in m.activeBody.units) { + val stmt = unit as Stmt + if (stmt is JAssignStmt) { + if (stmt.leftOp !is JimpleLocal) { + continue + } + val leftExpr = stmt.leftOp as JimpleLocal + val rightExpr = stmt.rightOp + if (leftExpr.name !== fragmentLocal.name) { + continue + } + var c: SootClass? = null + + // r7_4 = r3;r7_4=$r1; + if (rightExpr is JimpleLocal) { + c = Scene.v().getSootClassUnsafe(rightExpr.type.toString(), false) + + } else if (rightExpr is InvokeExpr) { + c = Scene.v() + .getSootClassUnsafe(resolveMethodException(rightExpr).returnType.toString(), false) + + } else if (rightExpr is JCastExpr) { + c = Scene.v().getSootClassUnsafe(rightExpr.op.type.toString(), false) + + if (c.isInterface) { + continue + } + } else if (rightExpr is FieldRef) { + c = Scene.v().getSootClassUnsafe(rightExpr.field.type.toString(), false) + + } + + if (c != null && Scene.v().orMakeFastHierarchy.isSubclass(c, androidFragmentClass)) { + addFragmentSootClassToSet(classes, c) + } + } + } + return classes + } + + private fun addFragmentSootClassToSet(s: MutableSet, c: SootClass) { + if (androidFragmentClassName == c.name) { + return + } + for (clz in s) { + if (clz.name === c.name) { + return + } + } + val subClassSet = HashSet() + PLUtils.getAllSubCLass(c, subClassSet) + s.addAll(subClassSet) + s.add(c) + } + + private fun findFragmentByLayoutFile(m: SootMethod): Set? { + val layoutId = findLayoutId(m) + return if (layoutId < 0) { + null + } else findFragmentsInLayout(layoutId) + + } + + + private fun findLayoutId(entry: SootMethod): Int { + val visited: MutableSet = HashSet() + return queryLayoutIdRecursive(entry, 4, visited) + } + + private fun queryLayoutIdRecursive( + entry: SootMethod, + depth: Int, + visited: MutableSet + ): Int { + if (depth <= 0) { + return -1 + } + if (visited.contains(entry.signature)) { + return -1 + } + visited.add(entry.signature) + val callees = ctx.callGraph.heirCallGraph[entry] ?: return -1 + if (!callees.contains(compatActivitySetContentView)) { + for (m in callees) { + val id = queryLayoutIdRecursive(m, depth - 1, visited) + if (id > 0) { + return id + } + } + } + for (unit in entry.activeBody.units) { + val stmt = unit as Stmt + if (stmt.containsInvokeExpr()) { + val invokeExpr = stmt.invokeExpr + if (compatActivitySetContentView.signature != resolveMethodException(invokeExpr).signature) { + continue + } + val value = invokeExpr.args[0] + if (value is IntConstant) { + return value.value + } + } + } + return -1 + } + + companion object { + fun processFragmentEntries(ctx: PreAnalyzeContext) { + val start = System.currentTimeMillis() + val f = Fragment(ctx) + for ((_, fakeEntry) in AndroidUtils.activityEntryMap) { + f.findFragmentAndAddCallbackToEntry(fakeEntry) + } + Log.logDebug("processFragmentEntries takes time " + (System.currentTimeMillis() - start) + "ms") + } + + + } + + private val compatActivitySetContentView: SootMethod = + Scene.v().grabMethod("") + + var fragmentTransactionMethods: HashMap = HashMap() + + + private var androidFragmentClass: SootClass = Scene.v().getSootClassUnsafe("androidx.fragment.app.Fragment", false) + + init { + mapOf( + "" to 1, + "" to 1, + "" to 0, + "" to 1, + "" to 1, + ).forEach { + val method = Scene.v().grabMethod(it.key) + fragmentTransactionMethods[method] = it.value + } + } +} diff --git a/src/main/kotlin/net/bytedance/security/app/KotlinEntry.kt b/src/main/kotlin/net/bytedance/security/app/KotlinEntry.kt new file mode 100644 index 0000000..694b092 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/KotlinEntry.kt @@ -0,0 +1,27 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app + +class KotlinEntry { + companion object { + @JvmStatic + fun callMain(args: Array) { + main(args) + } + } +} diff --git a/src/main/kotlin/net/bytedance/security/app/Log.kt b/src/main/kotlin/net/bytedance/security/app/Log.kt new file mode 100644 index 0000000..b518144 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/Log.kt @@ -0,0 +1,155 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +@file:Suppress("unused") + +package net.bytedance.security.app + +import java.io.File +import java.io.FileWriter +import java.io.IOException +import java.text.SimpleDateFormat +import java.util.* +import kotlin.system.exitProcess + +const val DEBUG = 0 +var INFO = 1 +var WARN = 2 +var ERROR = 3 + +/** + * A simple logger with buffering and delay writing. + */ +object Log { + + private var curLevel = INFO + + private val fileWriter: FileWriter + private val buffer: StringBuilder = StringBuilder() + private var lastTimeWrite: Long = System.currentTimeMillis() + private const val writeBufferInterval = 1000 //default delay time + const val TEXT_RESET = "\u001B[0m" + const val TEXT_BLACK = "\u001B[30m" + const val TEXT_RED = "\u001B[31m" + const val TEXT_GREEN = "\u001B[32m" + const val TEXT_YELLOW = "\u001B[33m" + const val TEXT_BLUE = "\u001B[34m" + const val TEXT_PURPLE = "\u001B[35m" + const val TEXT_CYAN = "\u001B[36m" + const val TEXT_WHITE = "\u001B[37m" + + init { + val out = getConfig().outPath + if (out.endsWith("/out/") || out.endsWith("/out")) { + println("this message should only appear in test case") +// Exception().printStackTrace() + } + val dirName = "$out/log/" + val dirFile = File(dirName) + if (dirFile.exists()) { + dirFile.delete() + } + if (!dirFile.exists()) { + dirFile.mkdirs() + } + val file = File(dirName, "main") + if (file.exists()) { + file.delete() + } + fileWriter = FileWriter(file, true) + } + + fun setLevel(level: Int) { + curLevel = level + } + + fun getLevelColor(level: Int): String { + when (level) { + WARN -> return TEXT_YELLOW + ERROR -> return TEXT_RED + } + return TEXT_RESET + } + + @Synchronized + private fun doLog(isLast: Boolean = false) { + val now = System.currentTimeMillis() + if (now - lastTimeWrite >= writeBufferInterval || isLast) { + try { + fileWriter.write(buffer.toString()) + } catch (e: IOException) { + e.printStackTrace() + exitProcess(14) + } + buffer.setLength(0) + lastTimeWrite = now + } + } + + private val df = SimpleDateFormat("yyyy-MM-dd HH:mm:ss:S") + fun logDebug(str: String) { + if (curLevel <= DEBUG) { + logStr(str, DEBUG) + } + } + + fun logInfo(str: String) { + if (curLevel <= INFO) { + logStr(str, INFO) + } + } + + fun logWarn(str: String) { + if (curLevel <= WARN) { + logStr(str, WARN) + } + } + + @Synchronized + fun logErr(str: String) { + logStr(str, ERROR) + } + + //Log and exit + fun logFatal(str: String) { + logErr(str) + flushAndClose() + exitProcess(-11) + } + + @Synchronized + fun flushAndClose() { + doLog(true) + fileWriter.flush() + fileWriter.close() + } + + fun logStr(str: String, level: Int) { + val day = Date() + val time = df.format(day) + buffer.append(time) + buffer.append(":") + buffer.append(str) + val color = getLevelColor(level) + println("$color$time:$str") +// println(buffer.toString()) + buffer.append("\n") + doLog() + } + + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/MethodFinder.kt b/src/main/kotlin/net/bytedance/security/app/MethodFinder.kt new file mode 100644 index 0000000..a4907cd --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/MethodFinder.kt @@ -0,0 +1,186 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app + +import net.bytedance.security.app.preprocess.MethodFieldConstCacheVisitor +import net.bytedance.security.app.util.methodSignatureDestruction +import net.bytedance.security.app.util.profiler +import net.bytedance.security.app.util.subSignature +import soot.Scene +import soot.SootClass +import soot.SootMethod + +/** + * find a specific method + */ +object MethodFinder { + /** + *@param pattern: a*, *a,aa*a,* + *@param target: string to match + */ + fun isMatched(pattern: String, target: String): Boolean { + var pattern2 = pattern + var target2 = target + if (pattern2 == "*") { + return true + } + pattern2 = pattern2.lowercase() + target2 = target2.lowercase() + return if (pattern2.startsWith("*") && pattern2.endsWith("*")) { + val patternStr = pattern2.split("\\*".toRegex()).toTypedArray()[1] + if (patternStr.isEmpty()) { + Log.logFatal("Format Error $pattern2") + } + target2.contains(patternStr) + } else if (pattern2.startsWith("*")) { + val partTargetStr = pattern2.split("\\*".toRegex()).toTypedArray()[1] + if (partTargetStr.isEmpty()) { + Log.logFatal("Format Error $pattern2") + } + target2.endsWith(partTargetStr) + } else if (pattern2.endsWith("*")) { + val partTargetStr = pattern2.split("\\*".toRegex()).toTypedArray()[0] + if (partTargetStr.isEmpty()) { + Log.logFatal("Format Error $pattern2") + } + target2.startsWith(partTargetStr) + } else { + target2 == pattern2 + } + } + + private fun addMatchedMethodSet(possibleMethodSigSet: MutableSet, sm: SootMethod) { + if (!MethodFieldConstCacheVisitor.canMethodHasSubMethods(sm)) { + possibleMethodSigSet.add(sm) + return + } + val sc = sm.declaringClass + val subClassSet = HashSet() + PLUtils.getAllSubCLass(sc, subClassSet) + for (sootClass in subClassSet) { + val subMethod = sootClass.getMethodUnsafe(sm.subSignature) + if (subMethod != null) { + possibleMethodSigSet.add(subMethod) + } + + } + } + + /** + * @param methodSig something like :<*: void onCreate(android.os.Bundle)> + * @return returns all matched methods + * prerequisite : + * 1. partial matching of class names is not supported, for example: com.security.Command* + * 2. If the class name is explicit, it will match all subclasses + */ + @Synchronized + private fun checkAndParseMethodSigInternal(methodSig: String): Set { + val matchedMethodSet: MutableSet = HashSet() + val fd = methodSignatureDestruction(methodSig) + if (!fd.className.contains("*") && !fd.functionName.contains("*") && !fd.args.contains("*") && !fd.returnType.contains( + "*" + ) + ) { + val sc = Scene.v().getSootClassUnsafe(fd.className, false) + val sm = Scene.v().grabMethod(methodSig) + if (sc != null && sm != null) { + matchedMethodSet.add(sm) + val subClassSet = HashSet() + PLUtils.getAllSubCLass(sc, subClassSet) + for (sootClass in subClassSet) { + val subMethod = sootClass.getMethodUnsafe(sm.subSignature) + if (subMethod != null) { + matchedMethodSet.add(subMethod) + } + } + } + + return matchedMethodSet + } + val targetClassSet: Collection + if (fd.className == "*") { + targetClassSet = Scene.v().classes + } else { + if (fd.className.indexOf("*") >= 0) { + throw Exception("invalid className $methodSig") + } + val sc = Scene.v().getSootClassUnsafe(fd.className, false) ?: return matchedMethodSet + targetClassSet = setOf(sc) + } + val possibleMethodSigSet: MutableSet = HashSet() + for (sc in targetClassSet) { + var methods: List + if (sc.name.startsWith(PLUtils.CUSTOM_CLASS)) { + continue + } else { + methods = sc.methods + } + if (fd.functionName.contains("*") || fd.args.contains("*") || fd.returnType.contains("*")) { + if (fd.functionName == "*") { + methods.forEach { + matchedMethodSet.add(it) + addMatchedMethodSet(possibleMethodSigSet, it) + } + } else { + for (sm in methods) { + if (isMatched(fd.functionName, sm.name)) { + matchedMethodSet.add(sm) + addMatchedMethodSet(possibleMethodSigSet, sm) + } + } + } + } else { + for (sm in methods) { + if (fd.subSignature() == sm.subSignature) { + matchedMethodSet.add(sm) + addMatchedMethodSet(possibleMethodSigSet, sm) + } + } + } + } + + for (otherSig in possibleMethodSigSet) { + matchedMethodSet.add(otherSig) + } + Log.logDebug(methodSig + " Parsed " + matchedMethodSet.size) + return matchedMethodSet + } + + /** + * cache for checkAndParseMethodSigInternal + */ + private var MethodSigMatchMapCache: MutableMap> = HashMap() + + + @Synchronized + fun checkAndParseMethodSig(methodSig: String): Set { + if (MethodSigMatchMapCache.containsKey(methodSig)) { + return MethodSigMatchMapCache[methodSig]!! + } + val start = System.currentTimeMillis() + val s = checkAndParseMethodSigInternal(methodSig) + profiler.checkAndParseMethodSigInternalTake(System.currentTimeMillis() - start) + MethodSigMatchMapCache[methodSig] = s + return s + } + + @Synchronized + fun clearCache() { + MethodSigMatchMapCache.clear() + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/PLUtils.kt b/src/main/kotlin/net/bytedance/security/app/PLUtils.kt new file mode 100644 index 0000000..00c2d23 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/PLUtils.kt @@ -0,0 +1,352 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app + +import net.bytedance.security.app.Log.logDebug +import net.bytedance.security.app.Log.logInfo +import soot.* +import soot.jimple.* +import java.io.FileWriter +import java.io.IOException +import java.io.PrintWriter +import java.util.concurrent.atomic.AtomicInteger + +object PLUtils { + const val LevelNormal = "normal" + const val LevelDanger = "dangerous" + const val LevelSig = "signature" + const val LevelSigOrSys = "signatureOrSystem" + + var JAVA_SRC = "/java/" + + var DATA_FIELD = "@data" + + var THIS_FIELD = "@this" + + + var PARAM = "@parameter" + + + var CONST_STR = "@const_str:" + const val CUSTOM_CLASS = "CustomClass" + + //entry method for whole program analyze + const val CUSTOM_CLASS_ENTRY = "<$CUSTOM_CLASS: void main()>" + const val CUSTOM_METHOD = "Main_Entry_" + + fun constStrSig(constant: String): String { + return CONST_STR + constant + } + + fun constSig(constant: Constant): String { + return if (constant is StringConstant) { + CONST_STR + constant.value + } else if (constant is NumericConstant) { + CONST_STR + constant.toString() + } else if (constant is NullConstant) { + CONST_STR + "null" + } else { + CONST_STR + constant.toString() + } + } + + + fun isStrMatch(pattern: String, target: String): Boolean { + val patternSub = pattern.replace("*", "") + return if (pattern.startsWith("*") && pattern.endsWith("*")) { + target.contains(patternSub) + } else if (pattern.startsWith("*")) { + target.endsWith(patternSub) + } else if (pattern.endsWith("*")) { + target.startsWith(patternSub) + } else { + target == patternSub + } + } + + fun writeFile(filePath: String, str: String) { + try { + val fw = FileWriter(filePath) + val out = PrintWriter(fw) + out.write(str) + out.println() + fw.close() + out.close() + } catch (e: IOException) { + e.printStackTrace() + } + } + + + /** + * get all subclass of sc, and save result to subClasses + */ + fun getAllSubCLass(sc: SootClass, subClasses: HashSet) { + if (sc.isInterface) { + val subClassSet = Scene.v().orMakeFastHierarchy.getAllImplementersOfInterface(sc) + if (subClassSet != null) { + for (sootClass in subClassSet) { + if (!subClasses.contains(sootClass)) { + subClasses.add(sootClass) + getAllSubCLass(sootClass, subClasses) + } + } + } + } else { + val subClassSet = Scene.v().orMakeFastHierarchy.getSubclassesOf(sc) + if (subClassSet != null) { + for (sootClass in subClassSet) { + if (!subClasses.contains(sootClass)) { + subClasses.add(sootClass) + getAllSubCLass(sootClass, subClasses) + } + } + } + } + } + + fun createCustomClass() { + val sClass = SootClass(CUSTOM_CLASS, Modifier.PUBLIC) + // 'extends Object' + sClass.superclass = Scene.v().getSootClass("java.lang.Object") + Scene.v().addClass(sClass) + } + + val entryId = AtomicInteger() + + + @Synchronized + fun createComponentEntry( + superClass: SootClass, + subClass: SootClass, + lifecycleMethods: List, + ): SootMethod { + val methodsToCall = ArrayList() + for (m in lifecycleMethods) { + val targetMethod = superClass.getMethodUnsafe(m) ?: continue + methodsToCall.add(targetMethod) + } + return entryMethod(subClass, methodsToCall, false) + } + + + @Synchronized + fun entryMethod(sc: SootClass, methodSet: List, preventDuplication: Boolean = true): SootMethod { + val className = CUSTOM_CLASS + // Declare 'public class classname' + val sClass = Scene.v().getSootClass(className) + // Create the method, public static void main(String[]) + val scName = sc.name.replace(".", "_").replace("$", "_") + var methodName = CUSTOM_METHOD + scName + if (preventDuplication) { + methodName += "_" + entryId.getAndIncrement() + } + val mainMethod = SootMethod( + methodName, + listOf(), + VoidType.v(), Modifier.PUBLIC or Modifier.STATIC + ) + sClass.methods.forEach { + if (it.name == methodName) { + return it + } + } + try { + sClass.addMethod(mainMethod) + logInfo("entryMethod addMethod ${mainMethod.signature}") + + // create empty body + val body = Jimple.v().newBody(mainMethod) + mainMethod.activeBody = body + val units = body.units + + + // Add some locals, component r0 + val instant: Local = Jimple.v().newLocal("r0", sc.type) + body.locals.add(instant) + // r1 = new component + val newExpr = Jimple.v().newNewExpr(sc.type) + val assignStmt = Jimple.v().newAssignStmt(instant, newExpr) + units.add(assignStmt) + val realMethodSet: ArrayList = ArrayList() + for (m in sc.methods) { + if (m.isConstructor) { + realMethodSet.add(m) + } + } + + realMethodSet.addAll(methodSet) + for (targetMethod in realMethodSet) { + val args: MutableList = ArrayList() + for (i in 0 until targetMethod.parameterCount) { + val argType = targetMethod.getParameterType(i) + val index = body.localCount + val localArg = Jimple.v().newLocal("v$index", argType) + body.locals.add(localArg) + args.add(localArg) + if (argType is PrimType) { + if (argType is FloatType) { + val argAssignStmt = Jimple.v().newAssignStmt(localArg, FloatConstant.v(3f)) + units.add(argAssignStmt) + } else if (argType is DoubleType) { + val argAssignStmt = Jimple.v().newAssignStmt(localArg, DoubleConstant.v(4.0)) + units.add(argAssignStmt) + } else { + val argAssignStmt = Jimple.v().newAssignStmt(localArg, IntConstant.v(5)) + units.add(argAssignStmt) + } + } else { + if (argType is ArrayType) { + val argNewExpr = Jimple.v().newNewArrayExpr(argType.baseType, IntConstant.v(2)) + val argAssignStmt = Jimple.v().newAssignStmt(localArg, argNewExpr) + units.add(argAssignStmt) + } else if (argType is RefType) { + val argNewExpr = Jimple.v().newNewExpr(argType) + val argAssignStmt = Jimple.v().newAssignStmt(localArg, argNewExpr) + units.add(argAssignStmt) + } + } + } + if (targetMethod.isStatic) { + // virtualinvoke target.(p0,p1,p2...); + val invokeStmt = Jimple.v().newInvokeStmt( + Jimple.v().newStaticInvokeExpr(targetMethod.makeRef(), args) + ) + units.add(invokeStmt) + } else { + // virtualinvoke target.(p0,p1,p2...); + var invokeStmt: InvokeStmt + try { + invokeStmt = Jimple.v() + .newInvokeStmt(Jimple.v().newVirtualInvokeExpr(instant, targetMethod.makeRef(), args)) + } catch (ex: Exception) { + invokeStmt = Jimple.v() + .newInvokeStmt(Jimple.v().newInterfaceInvokeExpr(instant, targetMethod.makeRef(), args)) + } + units.add(invokeStmt) + } + } + // insert "return" + units.add(Jimple.v().newReturnVoidStmt()) + return mainMethod + } catch (ex: Exception) { + ex.printStackTrace() + throw ex + } + } + + /** + * create virtual entry method for each class,if it doesn't have a caller + */ + private fun createTopMethodsCall(ctx: PreAnalyzeContext) { + for ((clz, methods) in ctx.callGraph.getTopMethods()) { + entryMethod(clz, methods.toList(), true) + } + } + + + fun createWholeProgramAnalyze(ctx: PreAnalyzeContext) { + createTopMethodsCall(ctx) + createCustomMainEntry() + } + + + private fun createCustomMainEntry(): SootMethod { + val className = CUSTOM_CLASS + // Declare 'public class classname' + val customClass = Scene.v().getSootClass(className) + // Create the method, public static void main(String[]) + val methodName = "main" + val mainMethod = SootMethod( + methodName, + listOf(), + VoidType.v(), Modifier.PUBLIC or Modifier.STATIC + ) + try { + customClass.addMethod(mainMethod) + logDebug("entryMethod addMethod ${mainMethod.signature}") + + // create empty body + val body = Jimple.v().newBody(mainMethod) + mainMethod.activeBody = body + val units = body.units + + for (targetMethod in customClass.methods) { + if (!targetMethod.isStatic) { + continue + } + if (targetMethod.name == "main") { + continue + } + + val invokeStmt = Jimple.v().newInvokeStmt( + Jimple.v().newStaticInvokeExpr(targetMethod.makeRef(), listOf()) + ) + units.add(invokeStmt) + } + // insert "return" + units.add(Jimple.v().newReturnVoidStmt()) + } catch (ex: Exception) { + ex.printStackTrace() + throw ex + } + return mainMethod + } + + fun dispatchCall(sootClass: SootClass, methodSubSig: String): SootMethod? { + var sootMethod = sootClass.getMethodUnsafe(methodSubSig) + if (sootMethod == null) { + sootMethod = if (sootClass.hasSuperclass()) { + dispatchCall(sootClass.superclass, methodSubSig) + } else { + return null + } + } + return sootMethod + } + + + fun dumpClass(className: String) { + val clz = Scene.v().getSootClassUnsafe(className, false) ?: return + println("class $className:") + val it = clz.methodIterator() + while (it.hasNext()) { + val m = it.next() + println(String.format("method:%s %s", m.signature, m.name)) + if (m.hasActiveBody()) { + println(String.format("%s", m.activeBody)) + } + } + } + + fun findMatchedChildClasses(targetSet: Set): MutableSet { + val findMatchedClasses: MutableSet = HashSet() + for (sc in Scene.v().classes) { + if (sc.hasSuperclass() && targetSet.contains(sc.superclass.name)) { + findMatchedClasses.add(sc) + continue + } + for (intf in sc.interfaces) { + if (targetSet.contains(intf.name)) { + findMatchedClasses.add(sc) + } + } + } + return findMatchedClasses + } +} diff --git a/src/main/kotlin/net/bytedance/security/app/PreAnalyzeContext.kt b/src/main/kotlin/net/bytedance/security/app/PreAnalyzeContext.kt new file mode 100644 index 0000000..5faf2ef --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/PreAnalyzeContext.kt @@ -0,0 +1,199 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app + +import net.bytedance.security.app.preprocess.* +import net.bytedance.security.app.rules.IRulesForContext +import net.bytedance.security.app.util.profiler +import soot.Scene +import soot.SootClass +import soot.SootField +import soot.SootMethod +import java.util.concurrent.atomic.AtomicInteger + +/** + * for jsb methods + */ +interface ContextWithJSBMethods { + fun getJSBMethods(): List +} + +/** + * The context before the pointer analysis ,it contains all the preprocessing information for the Java program. + */ + +open class PreAnalyzeContext { + + /** + * key is the function that is callee,value is the caller functions and the statement that occurs + * Direct is the function that is called directly without considering CHA relationship + * Heir is after considering CHA + */ + val methodDirectRefs: MutableMap> = HashMap() + + /** + * Key is the field to be loaded, and value is the callsite + * for example a=b.c; + */ + val loadFieldRefs: MutableMap> = HashMap() + + /** + *Key is the field to be stored, and value is the callSite + * for example a.b=c; + */ + val storeFieldRefs: MutableMap> = HashMap() + + /** + key is the pattern in the rule, and value is the place where possible matching constant strings appear + */ + var constStringPatternMap: MutableMap> = HashMap() + + + val newInstanceRefs: MutableMap> = HashMap() + + + val callGraph = CallGraph() + + + private var classCounter: AtomicInteger = AtomicInteger(0) + private var methodsCounter: AtomicInteger = AtomicInteger(0) + + + fun addMethodCounter(): Int { + return methodsCounter.incrementAndGet() + } + + fun addClassCounter(): Int { + return classCounter.incrementAndGet() + } + + fun getMethodCounter(): Int { + return methodsCounter.get() + } + + fun getClassCounter(): Int { + return classCounter.get() + } + + suspend fun createContext( + rules: IRulesForContext, + callBackEnhance: Boolean + ) { + val cam = createClassAndMethodHandler(this) + addClassAndMethodVisitor(cam, rules, callBackEnhance) + cam.run() + profiler.initProcessMethodStatistics(getMethodCounter(), getClassCounter(), this) + } + + private fun createClassAndMethodHandler(ctx: PreAnalyzeContext): AnalyzePreProcessor { + return AnalyzePreProcessor(getConfig().maxThread, ctx) + } + + protected open fun addClassAndMethodVisitor( + cam: AnalyzePreProcessor, rules: IRulesForContext, + callBackEnhance: Boolean + ) { + +// val constStrPatternInRules = MethodFieldConstCacheVisitor.parseAllConstStrPatternInRules(ruleDir, ruleList) + cam.addMethodVisitor { + //1. ssa Make sure SSA is at the first + MethodSSAVisitor() + }.addMethodVisitor { + //2. The callback must be handled after the SSA, otherwise the function doesn't have body + MethodCallbackVisitor(callBackEnhance) + }.addMethodVisitor { + //3. MethodFieldConstCacheVisitor must be handled after ssa, because there are dependencies + MethodFieldConstCacheVisitor( + this, + MethodStmtFieldCache(), + rules.constStringPatterns(), + rules.fields(), + rules.newInstances() + ) + }.addMethodVisitor { + MethodCounter(this) + } + cam.addClassVisitor { ClassCounter(this) } + } + + @Suppress("unused", "unused") + fun queryAMethod(method: SootMethod, result: MutableMap>, depth: Int) { + if (depth <= 0) { + return + } + if (result.containsKey(method)) { + return + } + if (callGraph.heirReverseCallGraph.containsKey(method)) { + result[method] = callGraph.heirReverseCallGraph[method]!! + for (m in callGraph.heirReverseCallGraph[method]!!) { + queryAMethod(m, result, depth - 1) + } + } + } + + + @Suppress("unused") + fun findInvokeCallSite(methodSig: String): Set { + val m = Scene.v().getMethod(methodSig) + return findInvokeCallSite(m) + } + + fun findInvokeCallSite(method: SootMethod): Set { + return this.methodDirectRefs[method] ?: setOf() + } + + fun findConstStringPatternCallSite(patternStr: String): Set { + return this.constStringPatternMap[patternStr] ?: setOf() + } + + fun findInstantCallSite(className: String): Set { + val clz = Scene.v().getSootClassUnsafe(className) ?: return emptySet() + return findInstantCallSite(clz) + } + + fun findInstantCallSite(clz: SootClass): Set { + return this.newInstanceRefs[clz] ?: setOf() + } + + fun findInstantCallSiteWithSubclass(className: String): Set { + val s = HashSet() + for (sc in Scene.v().classes) { + if (sc.name == className || sc.hasSuperclass() && className == sc.superclass.name) { + s.addAll(findInstantCallSite(sc)) + } + } + return s + } + + + /** + * field load callsites + */ + fun findFieldCallSite(field: String): Set { + val f = Scene.v().grabField(field) ?: return emptySet() + return findFieldCallSite(f) + } + + /** + * field load callsites + */ + fun findFieldCallSite(field: SootField): Set { + return this.loadFieldRefs[field] ?: setOf() + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/RuleData.kt b/src/main/kotlin/net/bytedance/security/app/RuleData.kt new file mode 100644 index 0000000..f12115d --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/RuleData.kt @@ -0,0 +1,323 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +@file:Suppress("LocalVariableName") + +package net.bytedance.security.app + +import kotlinx.serialization.KSerializer +import kotlinx.serialization.SerialName +import kotlinx.serialization.Serializable +import kotlinx.serialization.descriptors.PrimitiveKind +import kotlinx.serialization.descriptors.PrimitiveSerialDescriptor +import kotlinx.serialization.descriptors.SerialDescriptor +import kotlinx.serialization.encoding.Decoder +import kotlinx.serialization.encoding.Encoder +import kotlinx.serialization.json.* +import kotlinx.serialization.serializer +import net.bytedance.security.app.engineconfig.IOData +import net.bytedance.security.app.util.Json + +@Serializable +data class RuleDescription( + val category: String = "unknownCategory", + val detail: String? = null, + val model: String? = null, + val name: String = "unknownRule", + val possibility: String? = null, + val wiki: String? = null, + var level: String? = null, + val complianceCategory: String? = null, + val complianceCategoryDetail: String? = null, +) + + +/** + * "TaintTweak": { +"DisableEngineWrapper": true, +"MethodSignature": { +"": { +"@this->ret": { +"I": ["@this"], +"O": ["ret"] +} +} +} +} + */ +@Serializable +class TaintTweakData( + val DisableEngineWrapper: Boolean? = null, + val MethodName: Map>? = null, + val MethodSignature: Map>? = null +) + + +@Serializable +data class RuleData( + val TaintTweak: TaintTweakData? = null, + val maxSdk: Int? = null, + val desc: RuleDescription, + val enable: Boolean? = null, + val entry: Entry? = null, + var sink: Map? = null, + var kins: Map? = null, + var activeWhen: List? = null, + var noActiveWhen: List? = null, + val source: SourceBody? = null, + val sinkRuleObj: List? = null, + val sourceRuleObj: List? = null, + val throughAPI: ThroughAPI? = null, + val traceDepth: Int? = null, + val PrimTypeAsTaint: Boolean? = null, + val pathCnt: Int? = null, + val PolymorphismBackTrace: Boolean? = null, + val upperCross: Boolean? = null, + val debugLevel: Int? = null, + val printCG: Boolean? = null, + val IntentSerialTaintsAll: Boolean? = null, + + val sanitize: Map>? = null, + + val ManifestCheckMode: Boolean? = null, + val APIMode: Boolean? = null, + + val InternalMediaLocation: Boolean? = null, + + val FindDeeplinkMode: Boolean? = null, + val FieldSetMode: Boolean? = null, + val AdsCommandsList: JsonElement? = null, + val SplitterList: JsonElement? = null, + val AdsCommandsParameterList: JsonElement? = null, + val SmartRouterList: JsonElement? = null, + val XiGuaDeeplink: JsonElement? = null, + val SmartRouterParameterList: JsonElement? = null, + val CommonDeeplink: JsonElement? = null, + + val MethodMatchMode: Boolean? = null, + val excludeSdk: Boolean? = null, + + val FindClassMode: Boolean? = null, + val FindClass: FindClass? = null, + + val SliceMode: Boolean? = null, + val DirectMode: Boolean? = null, + val ConstStringMode: Boolean? = null, + val constLen: Int? = null, + val minLen: Int? = null, + val targetStringArr: List? = null, + + val ConstNumberMode: Boolean? = null, + val targetNumberArr: List? = null, +) + +val defaultSourceReturn = SourceReturn() + +@Serializable +data class SourceReturn( + + val EntryInvoke: Boolean = false, val LibraryOnly: Boolean? = false +) + +@Serializable +class SourceBody( + var Return: JsonElement? = null, + var UseJSInterface: Boolean = false, + var Param: Map> = mutableMapOf(), + var StaticField: List = mutableListOf(), + var ConstString: List = mutableListOf(), + var NewInstance: List = mutableListOf(), + var RuleObjReturn: List = mutableListOf(), + var Field: List = mutableListOf(), +) { + fun parseReturn(): Map { + var r = HashMap() + if (this.Return == null) { + return r + } + if (this.Return is JsonArray) { + val returns = Json.decodeFromJsonElement>(this.Return as JsonArray) + returns.forEach { r[it] = defaultSourceReturn } + } else { + r = Json.decodeFromJsonElement(this.Return!!) + } + RuleObjReturn.forEach { + if (!r.contains(it)) r[it] = defaultSourceReturn + } + return r + } +} + +@Serializable +data class RuleObjBody( + val ruleFile: String? = null, + val include: List = emptyList(), + val fromAPIMode: Boolean = false +) + +@Serializable(with = SinkBodySerializer::class) +class SinkBody( + val TaintCheck: List? = null, + val NotTaint: List? = null, + val LibraryOnly: Boolean? = null, + val TaintParamType: List? = null, + @SerialName("p*") val pstar: List? = null, + // p0,p1,p2.... + var pmap: Map>? = null, + //from:source,sink,both + val from: String? = null +) { + fun isEmpty(): Boolean { + return TaintCheck == null && NotTaint == null && TaintParamType == null && pstar == null && (pmap == null || pmap!!.isEmpty()) + } +} + +object SinkBodySerializer : KSerializer { + override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("SinkBody", PrimitiveKind.STRING) + + override fun serialize(encoder: Encoder, value: SinkBody) { + val element = buildJsonObject { + put("TaintCheck", Json.encodeToJsonElement(value.TaintCheck)) + value.NotTaint?.let { + put("NotTaint", Json.encodeToJsonElement(it)) + } + value.LibraryOnly?.let { + put("LibraryOnly", Json.encodeToJsonElement(it)) + } + value.TaintParamType?.let { + put("TaintParamType", Json.encodeToJsonElement(it)) + } + value.pstar?.let { + put("p*", Json.encodeToJsonElement(it)) + } + value.pmap?.forEach { + put(it.key, Json.encodeToJsonElement(it.value)) + } + } + encoder.encodeSerializableValue(serializer(), element) + } + + override fun deserialize(decoder: Decoder): SinkBody { + require(decoder is JsonDecoder) // this class can be decoded only by Json + val element = decoder.decodeJsonElement() + val m = element.jsonObject.toMutableMap() + var TaintCheck: List? = null + var from = "source" + m.remove("TaintCheck")?.let { + TaintCheck = Json.decodeFromJsonElement(it) + } + var NotTaint: List? = null + m.remove("NotTaint")?.let { + NotTaint = Json.decodeFromJsonElement(it) + } + var TaintParamType: List? = null + m.remove("TaintParamType")?.let { + TaintParamType = Json.decodeFromJsonElement(it) + } + m.remove("from")?.let { + from = Json.decodeFromJsonElement(it) + } + var pstar: List? = null + m.remove("p*")?.let { + pstar = Json.decodeFromJsonElement(it) + } + var libraryOnly: Boolean? = null + m.remove("LibraryOnly")?.let { + libraryOnly = Json.decodeFromJsonElement(it) + } + val pmap = HashMap>() + m.forEach { + pmap[it.key] = it.value.jsonArray + } + + return SinkBody( + TaintCheck = TaintCheck, + TaintParamType = TaintParamType, + NotTaint = NotTaint, + pstar = pstar, + LibraryOnly = libraryOnly, + pmap = pmap, + from = from + ) + } +} + + +@Serializable(with = KinsBodySerializer::class) +data class KinsBody( + val clz: String? = null, + // p0,p1,p2.... + var pmap: Map? = null +) + +object KinsBodySerializer : KSerializer { + override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("KinsBody", PrimitiveKind.STRING) + + override fun serialize(encoder: Encoder, value: KinsBody) { + val element = buildJsonObject { + put("clz", Json.encodeToJsonElement(value.clz)) + value.pmap?.forEach { + put(it.key, Json.encodeToJsonElement(it.value)) + } + } + encoder.encodeSerializableValue(serializer(), element) + } + + override fun deserialize(decoder: Decoder): KinsBody { + require(decoder is JsonDecoder) // this class can be decoded only by Json + val element = decoder.decodeJsonElement() + val m = element.jsonObject.toMutableMap() + var clz: String? = null + m.remove("clz")?.let { + clz = Json.decodeFromJsonElement(it) + } + val pmap = HashMap() + m.forEach { + pmap[it.key] = it.value.toString().replace("\"", "") + } + + return KinsBody( + clz = clz, + pmap = pmap, + ) + } +} + + +@Serializable +data class Entry( + val methods: List? = null, + val components: List? = null, + val UseJSInterface: Boolean? = null, + val ExportedCompos: Boolean? = null +) + +@Serializable +data class FindClass( + @SerialName("super") val Super: List? = null, + @SerialName("implements") val Implements: List? = null, + val methods: Map? = null +) + +@Serializable +data class FindClassMethodsExclude(val exclude: Set? = null, val include: Set? = null) + +@Serializable +data class ThroughAPI( + val MethodName: List? = null, + val MethodSignature: List? = null +) diff --git a/src/main/kotlin/net/bytedance/security/app/SliceAnalyzeWrapper.kt b/src/main/kotlin/net/bytedance/security/app/SliceAnalyzeWrapper.kt new file mode 100644 index 0000000..1cd3bbc --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/SliceAnalyzeWrapper.kt @@ -0,0 +1,122 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app + +import net.bytedance.security.app.Log.logInfo +import net.bytedance.security.app.engineconfig.EngineConfig +import net.bytedance.security.app.pathfinder.TaintPathFinder +import net.bytedance.security.app.pointer.PointerFactory +import net.bytedance.security.app.rules.TaintFlowRule +import net.bytedance.security.app.taintflow.* +import net.bytedance.security.app.util.TaskQueue +import net.bytedance.security.app.util.profiler +import java.util.concurrent.atomic.AtomicInteger + + +data class AnalyzersAndDepth( + val analyzers: ArrayList, + var depth: Int, + val rule: TaintFlowRule, + val name: String +) + + +class SliceAnalyzeWrapper( + val ctx: PreAnalyzeContext, + val analyzers: List, +) { + private fun groupAnalyzers(): Map { + val m = HashMap() + for (a in analyzers) { + val name = "${a.entryMethod.signature}- ${a.rule.name}" + val l = + m.computeIfAbsent(name) { + AnalyzersAndDepth( + ArrayList(), + 0, + a.rule, + name + ) + } + l.analyzers.add(a) + if (a.thisDepth > l.depth) { + l.depth = a.thisDepth + } + } + + return m + } + + suspend fun createPathFinderQueue(): TaskQueue { + val q = TaskQueue("TaintPathFinder", getConfig().maxThread) { finder, _ -> + finder.analyze() + profiler.stopTaintPathCalc(finder.analyzer.entryMethod.signature) + } + return q + } + + suspend fun run() { + val defaultPointerPropagationRule = DefaultPointerPropagationRule(EngineConfig.PointerPropagationConfig) + val defaultVariableFlowRule = DefaultVariableFlowRule(EngineConfig.variableFlowConfig) + val allTask = groupAnalyzers() + val finishedTask = AtomicInteger(0) + val finderQueue = createPathFinderQueue() + val analyzeTimeInSeconds = + getConfig().maxPointerAnalyzeTime * 1000.toLong() / 3 * 2 + val q = TaskQueue("SliceAnalyzeWrapper", getConfig().maxThread) { ad, _ -> + val vfr = if (ad.rule.taintTweak != null) { + TaintTweakTaintFlowRule(ad.rule.taintTweak, defaultVariableFlowRule) + } else { + defaultVariableFlowRule + } + val methodAnalyzeMode = if (getConfig().skipAnalyzeNonRelatedMethods) { + PruneMethodAnalyzeMode.fromTaintAnalyzers(analyzers, ad.depth, ctx) + } else { + DefaultMethodAnalyzeMode + } + val tsp = TwoStagePointerAnalyze( + ad.name, + ad.analyzers.first().entryMethod, AnalyzeContext(PointerFactory()), ad.depth, + defaultPointerPropagationRule, + vfr, + methodAnalyzeMode, + analyzeTimeInSeconds + ) + tsp.doPointerAnalyze() + profiler.entryContext(ad.name, tsp.ctx) + profiler.startTaintPathCalc(ad.analyzers.first().entryMethod.signature) + for (analyzer in ad.analyzers) { + val finder = TaintPathFinder(ctx, tsp.ctx, analyzer.rule, analyzer) + finderQueue.addTask(finder) + } + val n = finishedTask.addAndGet(1) + logInfo("${this.javaClass.simpleName} finished $n/${allTask.size}") + } + val finderQueueJob = finderQueue.runTask() + + val job = q.runTask() + for ((_, ads) in allTask) { + q.addTask(ads) + } + q.addTaskFinished() + job.join() + + finderQueue.addTaskFinished() + finderQueueJob.join() + } +} diff --git a/src/main/kotlin/net/bytedance/security/app/StaticAnalyzeMain.kt b/src/main/kotlin/net/bytedance/security/app/StaticAnalyzeMain.kt new file mode 100644 index 0000000..8e104bd --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/StaticAnalyzeMain.kt @@ -0,0 +1,104 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app + +import kotlinx.coroutines.runBlocking +import net.bytedance.security.app.Fragment.Companion.processFragmentEntries +import net.bytedance.security.app.Log.logInfo +import net.bytedance.security.app.android.AndroidUtils +import net.bytedance.security.app.android.AndroidUtils.loadDynamicRegisterReceiver +import net.bytedance.security.app.android.AndroidUtils.parseApk +import net.bytedance.security.app.engineconfig.EngineConfig +import net.bytedance.security.app.util.Json +import net.bytedance.security.app.util.profiler +import java.nio.file.Files +import java.nio.file.Paths + + +object StaticAnalyzeMain { + @Throws(Exception::class) + suspend fun startAnalyze(argumentConfig: ArgumentConfig) { + val apkPath = argumentConfig.apkPath + val v3 = AnalyzeStepByStep() + val jadxPath = "${argumentConfig.configPath}/tools/jadx/bin/jadx" + val apkNameTool = "${argumentConfig.configPath}/tools/ApkName.sh" + + logInfo("started...") + profiler.startMemoryProfile() + v3.initSoot( + AnalyzeStepByStep.TYPE.APK, + apkPath, + "${argumentConfig.configPath}/tools/platforms", + argumentConfig.outPath + ) + logInfo("soot init done") + PLUtils.createCustomClass() + profiler.parseApk.start() + parseApk(apkPath, jadxPath, argumentConfig.outPath, apkNameTool) + logInfo("apk parse done") + profiler.parseApk.end() + + profiler.preProcessor.start() + val rules = v3.loadRules(argumentConfig.rules.split(",")) + logInfo("rules loaded") + val ctx = v3.createContext(rules) + profiler.preProcessor.end() + + + if (getConfig().doWholeProcessMode) { + PLUtils.createWholeProgramAnalyze(ctx) + } + loadDynamicRegisterReceiver(ctx) + + if (argumentConfig.supportFragment) { + profiler.fragments.start() + processFragmentEntries(ctx) + profiler.fragments.end() + } + AndroidUtils.initLifeCycle() + val analyzers = v3.parseRules(ctx, rules) + v3.solve(ctx, analyzers) + profiler.stopMemoryProfile() + } +} + +@Throws(Exception::class) +fun main(args: Array) { + if (args.isEmpty()) { + println("Usage: java -jar appshark.jar config.json5") + return + } + val configPath = args[0] + try { + val configJson = String(Files.readAllBytes(Paths.get(configPath))) + val argumentConfig: ArgumentConfig = Json.decodeFromString(configJson) + cfg = argumentConfig + ArgumentConfig.mergeWithDefaultConfig(argumentConfig) + Log.setLevel(argumentConfig.logLevel) + logInfo("welcome to appshark ${EngineInfo.Version}") + getConfig().libraryPackage?.let { + if (it.isNotEmpty()) { + EngineConfig.libraryConfig.setPackage(it) + } + } + runBlocking { StaticAnalyzeMain.startAnalyze(argumentConfig) } + } catch (e: Exception) { + e.printStackTrace() + } + Log.flushAndClose() +} diff --git a/src/main/kotlin/net/bytedance/security/app/WholeProcessAnalyzeWrapper.kt b/src/main/kotlin/net/bytedance/security/app/WholeProcessAnalyzeWrapper.kt new file mode 100644 index 0000000..36888ab --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/WholeProcessAnalyzeWrapper.kt @@ -0,0 +1,64 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app + +import net.bytedance.security.app.engineconfig.EngineConfig +import net.bytedance.security.app.pathfinder.TaintPathFinder +import net.bytedance.security.app.pointer.PointerFactory +import net.bytedance.security.app.taintflow.* +import net.bytedance.security.app.util.TaskQueue +import soot.Scene + +/** +whole program pointer analyze,, + */ +class WholeProcessAnalyzeWrapper( + val ctx: PreAnalyzeContext, + val analyzers: List, +) { + suspend fun run() { + val maxDepth = 3000 + val analyzeTimeInSeconds = getConfig().maxPointerAnalyzeTime * 1000.toLong() / 2 + val entry = Scene.v().getMethod(PLUtils.CUSTOM_CLASS_ENTRY) + val methodAnalyzeMode = if (getConfig().skipAnalyzeNonRelatedMethods) { + PruneMethodAnalyzeMode.fromTaintAnalyzers(analyzers, maxDepth, ctx) + } else { + DefaultMethodAnalyzeMode + } + val tsp = TwoStagePointerAnalyze( + "whole_process_analyze_main", + entry, AnalyzeContext(PointerFactory()), maxDepth, DefaultPointerPropagationRule( + EngineConfig.PointerPropagationConfig + ), + DefaultVariableFlowRule(EngineConfig.variableFlowConfig), + methodAnalyzeMode, + analyzeTimeInSeconds + ) + tsp.doPointerAnalyze() + val q = TaskQueue("WholeProcessPathFinder", getConfig().maxThread) { analyzer, _ -> + val finder = TaintPathFinder(ctx, tsp.ctx, analyzer.rule, analyzer) + finder.analyze() + } + val job = q.runTask() + for (analyzer in analyzers) { + q.addTask(analyzer) + } + q.addTaskFinished() + job.join() + } +} diff --git a/src/main/kotlin/net/bytedance/security/app/android/AndroidUtils.kt b/src/main/kotlin/net/bytedance/security/app/android/AndroidUtils.kt new file mode 100644 index 0000000..47fb64f --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/android/AndroidUtils.kt @@ -0,0 +1,606 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.android + +import kotlinx.serialization.KSerializer +import kotlinx.serialization.Serializable +import kotlinx.serialization.descriptors.PrimitiveKind +import kotlinx.serialization.descriptors.PrimitiveSerialDescriptor +import kotlinx.serialization.descriptors.SerialDescriptor +import kotlinx.serialization.encoding.Decoder +import kotlinx.serialization.encoding.Encoder +import kotlinx.serialization.json.buildJsonObject +import kotlinx.serialization.json.put +import kotlinx.serialization.serializer +import net.bytedance.security.app.* +import net.bytedance.security.app.result.model.AnySerializer +import net.bytedance.security.app.util.Json +import soot.RefType +import soot.Scene +import soot.SootClass +import soot.SootMethod +import soot.jimple.Constant +import soot.jimple.InstanceInvokeExpr +import soot.jimple.Stmt +import soot.jimple.infoflow.android.axml.AXmlHandler +import soot.jimple.infoflow.android.axml.AXmlNode +import soot.jimple.infoflow.android.axml.ApkHandler +import soot.jimple.infoflow.android.axml.parsers.AXML20Parser +import soot.jimple.infoflow.android.manifest.IAndroidApplication +import soot.jimple.infoflow.android.manifest.ProcessManifest +import soot.jimple.infoflow.android.manifest.binary.AbstractBinaryAndroidComponent +import soot.jimple.infoflow.android.resources.ARSCFileParser +import soot.jimple.infoflow.android.resources.LayoutFileParser +import java.io.File +import java.io.IOException +import java.nio.charset.StandardCharsets +import java.util.concurrent.TimeUnit +import java.util.zip.ZipEntry +import java.util.zip.ZipFile +import kotlin.system.exitProcess + +interface ManifestVulnerability { + fun checkDebugOrBackup(app: IAndroidApplication) + fun checkProviderMisConfigPath(aXmlNode: AXmlNode, isExported: Boolean, xmlInfo: ComponentDescription) +} + +/** + * for convenience to recognize a particular structure during serialization + */ +interface ToMapSerializeHelper { + fun toMap(): Map +} + +@Serializable(with = ComponentDescriptionDataSerializer::class) +class ComponentDescription : ToMapSerializeHelper, Cloneable { + var exported: Boolean = false + var trace: List? = null + var stringMap: MutableMap = HashMap() + var otherMap: MutableMap = HashMap() + override fun toMap(): Map { + val m = HashMap() + + m["exported"] = this.exported + this.trace?.let { + m["trace"] = this.trace!! + } + this.stringMap.forEach { + m[it.key] = it.value + } + this.otherMap.forEach { + m[it.key] = it.value + } + return m + } + + public override fun clone(): ComponentDescription { + val m2 = ComponentDescription() + m2.exported = this.exported + this.trace?.let { + m2.trace = ArrayList(it) + } + m2.stringMap = HashMap(this.stringMap) + m2.otherMap = HashMap(this.otherMap) + return m2 + } +} + + +object ComponentDescriptionDataSerializer : KSerializer { + override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("ComponentDescription", PrimitiveKind.STRING) + + override fun serialize(encoder: Encoder, value: ComponentDescription) { + val element = buildJsonObject { + put("exported", value.exported) + value.trace?.let { + put("trace", Json.encodeToJsonElement(it)) + } + value.stringMap.forEach { + put(it.key, it.value) + } + value.otherMap.forEach { + put(it.key, Json.encodeToJsonElement(AnySerializer, it.value)) + } + } + encoder.encodeSerializableValue(serializer(), element) + } + + override fun deserialize(decoder: Decoder): ComponentDescription { + error("deserialize is not supported") + } +} + +object AndroidUtils { + var apkAbsPath: String? = null + var JavaSourceDir: String? = null + + var dexToJavaProcess: Process? = null + var jadxAbsPath: String? = null + var resources: ARSCFileParser? = null + var isApkParsed = false // + + @Suppress("unused") + var exportDeeplinkCompos: MutableSet = HashSet() + + var exportComponents: MutableSet = HashSet() + var unExportComponents: MutableSet = HashSet() + var exportActivities: MutableSet = HashSet() + var unExportActivities: MutableSet = HashSet() + var exportServices: MutableSet = HashSet() + var unExportServices: MutableSet = HashSet() + var exportProviders: MutableSet = HashSet() + var unExportProviders: MutableSet = HashSet() + var exportReceivers: MutableSet = HashSet() + var unExportReceivers: MutableSet = HashSet() + + var compoEntryMap: MutableMap = HashMap() + + var dummyToDirectEntryMap: MutableMap = HashMap() + + // All Activities =>fake entry method + var activityEntryMap: MutableMap = HashMap() + + // The key and value in compoEntryMap are swapped + var entryCompoMap: MutableMap = HashMap() + + var compoXmlMapByType: MutableMap> = HashMap() + + var GlobalCompoXmlMap: MutableMap = HashMap() + var layoutFileParser: LayoutFileParser? = null + + /** + * user-defined permission + */ + var permissionMap: Map = HashMap() + + + var usePermissionSet: Set = HashSet() + + var PackageName: String = "" + + var ApplicationName: String = "" + + var AppLabelName: String = "" + + var VersionName: String = "" + + var VersionCode = 0 + + var MinSdk = 0 + + var TargetSdk = 0 + private var manifestVulnerability: ManifestVulnerability? = null + private fun dexToJava(apkPath: String, outPath: String) { + JavaSourceDir = outPath + PLUtils.JAVA_SRC + val thread = Runtime.getRuntime().availableProcessors() / 2 + try { + val start = System.currentTimeMillis() + Log.logInfo("==========>Start dex to Java") + val processBuilder = ProcessBuilder( + "$jadxAbsPath.sh", + jadxAbsPath, + apkPath, + JavaSourceDir, thread.toString() + ) + Log.logInfo(processBuilder.command().toString()) + dexToJavaProcess = processBuilder.start() + dexToJavaProcess?.waitFor(1800, TimeUnit.SECONDS) + dexToJavaProcess?.destroy() + dexToJavaProcess?.waitFor() + Log.logInfo("Dex to Java Done " + (System.currentTimeMillis() - start) + "ms<==========") + } catch (e: Exception) { + e.printStackTrace() + } + Log.logInfo("write Java Source to $JavaSourceDir") + } + + + fun parseApk(apkPath: String, jadxPath: String, outPath: String, apkNameToolPath: String) { + try { + parseApkInternal(apkPath, jadxPath, outPath, apkNameToolPath) + } catch (ex: Exception) { + ex.printStackTrace() + } + } + + /** + * 1. Parse APK meta information + * 2. Convert dex to Java + * 3. Address manifest bugs + */ + private fun parseApkInternal(apkPath: String, jadxPath: String, outPath: String, apkNameToolPath: String) { + try { + val processBuilder = ProcessBuilder( + apkNameToolPath, + apkPath + ) + Log.logInfo(processBuilder.command().toString()) + val process = processBuilder.start() + process.waitFor() + val stream = process.inputStream + val out = ByteArray(128) + val ret = stream.read(out) + if (ret > 0) { + AppLabelName = String(out, StandardCharsets.UTF_8) + AppLabelName = AppLabelName.trim { it <= ' ' } + } + } catch (e: Exception) { + Log.logErr("$apkPath -> $outPath") + e.printStackTrace() + exitProcess(29) + } + apkAbsPath = apkPath + if (getConfig().javaSource == true) { + Log.logDebug("Dex to java code") + jadxAbsPath = jadxPath + dexToJava(apkPath, outPath) + } + + val targetAPK = File(apkAbsPath!!) + Log.logDebug("Load resource") + resources = ARSCFileParser() + try { + resources!!.parse(targetAPK.absolutePath) + } catch (e: IOException) { + e.printStackTrace() + } + Log.logDebug("Load manifest") + val manifest: ProcessManifest = try { + ProcessManifest(targetAPK, resources) + } catch (e: Exception) { + e.printStackTrace() + try { + val apk = ApkHandler(targetAPK) + val manifestInputStream = apk.getInputStream("AndroidManifest.xml") + val aXmlHandler = AXmlHandler(manifestInputStream) + val manifests = aXmlHandler.getNodesWithTag("manifest") + if (manifests.size > 0) { + val manifest = manifests[0] + PackageName = manifest.getAttribute("package").value as String + Log.logDebug("package $PackageName") + } + } catch (ioException: IOException) { + ioException.printStackTrace() + exitProcess(31) + } + return + } + usePermissionSet = manifest.permissions + permissionMap = getDefinedPermissions(manifest.manifest) + Log.logDebug("use perm $usePermissionSet") + Log.logDebug("def perm $permissionMap") + PackageName = manifest.packageName + Log.logDebug("package $PackageName") + if (manifest.application.name != null) { + ApplicationName = manifest.application.name + } + Log.logDebug("ApplicationName $ApplicationName") + Log.logDebug("AppName $AppLabelName") + manifest.versionName?.let { + VersionName = manifest.versionName + } + Log.logDebug("VersionName $VersionName") + VersionCode = manifest.versionCode + Log.logDebug("VersionCode $VersionCode") + MinSdk = manifest.minSdkVersion + Log.logDebug("MinSdk $MinSdk") + TargetSdk = manifest.targetSdkVersion + Log.logDebug("TargetSdk $TargetSdk") + layoutFileParser = LayoutFileParser(manifest.packageName, resources) + layoutFileParser!!.parseLayoutFileDirect(apkPath) + parseAllComponents(manifest) + + this.manifestVulnerability?.checkDebugOrBackup(manifest.application) + isApkParsed = true + } + + fun getDefinedPermissions(manifestAxml: AXmlNode): Map { + val tmpPermissionMap: MutableMap = HashMap() + val usesPerms = manifestAxml.getChildrenWithTag("permission") + val iterator = usesPerms.iterator() + while (iterator.hasNext()) { + val perm = iterator.next() as AXmlNode + val name = perm.getAttribute("name") + try { + val protectionLevel = perm.getAttribute("protectionLevel") + if (name != null) { + val permission = name.value as String + if (protectionLevel != null) { + val level = when (protectionLevel.value) { + is Int -> { + protectionLevel.value + } + is String -> { + try { + (protectionLevel.value as String).toInt() + } catch (ex: Exception) { + ex.printStackTrace() + 0 + } + } + else -> { + 0 + } + } + var protection = PLUtils.LevelNormal + when (level) { + 0 -> protection = PLUtils.LevelNormal + 1 -> protection = PLUtils.LevelDanger + 2 -> protection = PLUtils.LevelSig + 3 -> protection = PLUtils.LevelSigOrSys + } + tmpPermissionMap[permission] = protection + } else { + tmpPermissionMap[permission] = PLUtils.LevelNormal + } + } + } catch (ex: Exception) { + ex.printStackTrace() + } + } + return tmpPermissionMap + } + + fun loadDynamicRegisterReceiver(ctx: PreAnalyzeContext) { + val broadcastRecSig = + "" + val broadcastRecName = "registerReceiver" + val methodSet = MethodFinder.checkAndParseMethodSig(broadcastRecSig) + val invokeMap = ctx.methodDirectRefs + for (method in methodSet) { + val callerSet = invokeMap[method] ?: continue + for (sm in callerSet) { + if (!sm.method.isConcrete) { + continue + } + for (unit in sm.method.retrieveActiveBody().units) { + val stmt = unit as Stmt + if (!stmt.containsInvokeExpr()) { + continue + } + val invokeExpr = stmt.invokeExpr + if (invokeExpr.argCount < 2) { + continue + } + if (invokeExpr is InstanceInvokeExpr) { + if (invokeExpr.getMethodRef().name == broadcastRecName) { + val arg0 = invokeExpr.getArgs()[0] + if (arg0 is Constant) { + continue + } + if (Scene.v().orMakeFastHierarchy.canStoreType( + arg0.type, + RefType.v("android.content.BroadcastReceiver") + ) + ) { + val className = arg0.type.toString() + if (className == "android.content.BroadcastReceiver") { + continue + } + val sc = Scene.v().getSootClassUnsafe(className, false) + if (sc != null) { +// PLLog.logErr("add BroadcastReceiver: "+className+" => "+stmt.toString()); + exportReceivers.add(sc) + exportComponents.add(sc) + val broadcastReceiver = ComponentDescription() + broadcastReceiver.stringMap["DynamicBroadcastReceiver"] = className + broadcastReceiver.stringMap["RegisteredMethod"] = sm.method.signature + broadcastReceiver.stringMap["RegisteredStmt"] = stmt.toString() + broadcastReceiver.exported = true + if (!compoXmlMapByType.containsKey("exportedReceivers")) { + compoXmlMapByType["exportedReceivers"] = HashMap() + } + compoXmlMapByType["exportedReceivers"]!![className] = broadcastReceiver + GlobalCompoXmlMap[className] = broadcastReceiver + } + } + } + } + } + } + } + } + + @Synchronized + fun buildEntryCompoMap() { + for ((key, value) in compoEntryMap) { + entryCompoMap[value] = key + } + } + + fun initLifeCycle() { + Log.logInfo("exportActivities=$exportActivities") + for (compo in exportActivities) { + val entry = + PLUtils.createComponentEntry(LifecycleConst.ActivityClass, compo, LifecycleConst.ActivityMethods) + compoEntryMap[compo] = entry + activityEntryMap[compo] = entry + } + for (compo in unExportActivities) { + val entry = + PLUtils.createComponentEntry(LifecycleConst.ActivityClass, compo, LifecycleConst.ActivityMethods) + compoEntryMap[compo] = entry + activityEntryMap[compo] = entry + } + for (compo in exportReceivers) { + val entry = PLUtils.createComponentEntry( + LifecycleConst.BroadcastReceiverClass, + compo, + LifecycleConst.BroadcastReceiverMethods + ) + compoEntryMap[compo] = entry + } + for (compo in unExportReceivers) { + val entry = PLUtils.createComponentEntry( + LifecycleConst.BroadcastReceiverClass, + compo, + LifecycleConst.BroadcastReceiverMethods + ) + compoEntryMap[compo] = entry + } + for (compo in exportServices) { + val entry = PLUtils.createComponentEntry(LifecycleConst.ServiceClass, compo, LifecycleConst.ServiceMethods) + compoEntryMap[compo] = entry + } + for (compo in unExportServices) { + val entry = PLUtils.createComponentEntry(LifecycleConst.ServiceClass, compo, LifecycleConst.ServiceMethods) + compoEntryMap[compo] = entry + } + for (compo in exportProviders) { + val entry = + PLUtils.createComponentEntry(LifecycleConst.ContentProviderClass, compo, LifecycleConst.ProviderMethods) + compoEntryMap[compo] = entry + } + for (compo in unExportProviders) { + val entry = + PLUtils.createComponentEntry(LifecycleConst.ContentProviderClass, compo, LifecycleConst.ProviderMethods) + compoEntryMap[compo] = entry + } + buildEntryCompoMap() + } + + + fun parseResAXml(fileName: String): AXmlNode? { + val apkF = File(apkAbsPath!!) + if (!apkF.exists()) { + Log.logDebug("file '$apkAbsPath' does not exist!") + return null + } + try { + ZipFile(apkF).use { archive -> + val entries = archive.entries() + while (entries.hasMoreElements()) { + val entry = entries.nextElement() as ZipEntry + val entryName = entry.name + if (entryName == fileName) { + archive.getInputStream(entry).use { inputStream -> + val handler = AXmlHandler(inputStream, AXML20Parser()) + return handler.document.rootNode + } + } + } + } + } catch (e: Exception) { + Log.logErr("Error when looking for XML resource files in apk $apkAbsPath") + } + return null + } + + private fun parseComponent( + c: AbstractBinaryAndroidComponent, + exportedCompoSet: MutableSet, + unExportedCompoSet: MutableSet, + type: String + ): Boolean { + val aXmlNode = c.aXmlNode + val xmlInfo = ComponentDescription() + var isExportedCompo = c.isExported + val enabled = c.isEnabled + if (enabled) { + val childNodes = aXmlNode.getChildrenWithTag("intent-filter") + if (childNodes.isNotEmpty()) { + isExportedCompo = true + } + } + + if (isExportedCompo) { + if (aXmlNode.hasAttribute("permission")) { + val perm = aXmlNode.getAttribute("permission").value as String + Log.logDebug("perm $perm") + if (permissionMap.containsKey(perm)) { + Log.logDebug("level " + permissionMap[perm]) + if (permissionMap[perm] != PLUtils.LevelNormal) { + isExportedCompo = false + } + } else { + isExportedCompo = false + } + } + } + val classNameObj = c.nameString ?: return false // aXmlNode.getAttribute("name")?.value + val className: String = classNameObj + val sc = Scene.v().getSootClassUnsafe(className, false) ?: return false + val key = aXmlNode.toString().replace("\"", "") + val itemArr: MutableList>> = ArrayList() + + for (node in aXmlNode.children) { + val itemKey = node.toString().replace("\"", "") + val item: MutableMap> = HashMap() + itemArr.add(item) + val itemValueList: MutableList = ArrayList() + item[itemKey] = itemValueList + for (itemNode in node.children) { + itemValueList.add(itemNode.toString().replace("\"", "")) + } + } + xmlInfo.otherMap[key] = Json.encodeToJsonElement(itemArr) + val compoKey: String + if (isExportedCompo) { + compoKey = "exported$type" + exportedCompoSet.add(sc) + exportComponents.add(sc) + xmlInfo.exported = true + } else { + compoKey = "unExported$type" + unExportedCompoSet.add(sc) + unExportComponents.add(sc) + xmlInfo.exported = false + } + if (type == "Providers") { + this.manifestVulnerability?.checkProviderMisConfigPath(aXmlNode, isExportedCompo, xmlInfo) + } + if (!compoXmlMapByType.containsKey(compoKey)) { + compoXmlMapByType[compoKey] = HashMap() + } + compoXmlMapByType[compoKey]!![className] = xmlInfo + GlobalCompoXmlMap[className] = xmlInfo + return isExportedCompo + } + + private fun parseAllComponents(manifests: ProcessManifest) { + for (aXmlNode in manifests.activities) { + parseComponent(aXmlNode, exportActivities, unExportActivities, "Activities") + } + for (aXmlNode in manifests.broadcastReceivers) { + parseComponent(aXmlNode, exportReceivers, unExportReceivers, "Receivers") + } + for (aXmlNode in manifests.contentProviders) { + parseComponent(aXmlNode, exportProviders, unExportProviders, "Providers") + } + for (aXmlNode in manifests.services) { + parseComponent(aXmlNode, exportServices, unExportServices, "Services") + } + } + + /** + layoutId corresponding to R.layout.XXX + the return may be null. + */ + fun findFragmentsInLayout(layoutId: Int): Set? { + val r = resources!!.findResource(layoutId) + val name = r.resourceName + // System.out.println(String.format("getResourceName:%s", name)); + val key = "res/layout/$name.xml" + val fragments = layoutFileParser!!.fragments + return fragments[key] + } + + fun setManifestVulnerability(manifestVulnerability: ManifestVulnerability) { + this.manifestVulnerability = manifestVulnerability + } +} diff --git a/src/main/kotlin/net/bytedance/security/app/android/LifecycleConst.kt b/src/main/kotlin/net/bytedance/security/app/android/LifecycleConst.kt new file mode 100644 index 0000000..166ec16 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/android/LifecycleConst.kt @@ -0,0 +1,147 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.android + +import soot.Scene +import soot.SootClass + +/* +Calculate the inheritable functions of the android component classes to +facilitate the automatic generation of Entry functions + */ +object LifecycleConst { + var ActivityClass: SootClass = Scene.v().getSootClassUnsafe( + "android.app.Activity", false + ) + var ServiceClass: SootClass = Scene.v().getSootClassUnsafe( + "android.app.Service", false + ) + + @Suppress("unused") + var IntentServiceClass: SootClass = Scene.v().getSootClassUnsafe( + "android.app.IntentService", false + ) + var BroadcastReceiverClass: SootClass = Scene.v().getSootClassUnsafe( + "android.content.BroadcastReceiver", false + ) + var ContentProviderClass: SootClass = Scene.v().getSootClassUnsafe( + "android.content.ContentProvider", false + ) + var FragmentClass: SootClass? = Scene.v().getSootClassUnsafe( + "androidx.fragment.app.Fragment", false + ) + + /** + *all the overridable methods of android.app.Activity + */ + val ActivityMethods = listOf( + "void onBackPressed()", + "void onCreate(android.os.Bundle)", + "void onDestroy()", + "void onPause()", + "void onRestart()", + "void onResume()", + "void onStart()", + "void onStop()", + "void onSaveInstanceState(android.os.Bundle)", + "void onRestoreInstanceState(android.os.Bundle)", + "java.lang.CharSequence onCreateDescription()", + "void onPostCreate(android.os.Bundle)", + "void onPostResume()", + "void onAttachFragment(android.app.Fragment)" + ) + + /** + * all the overridable methods of android.app.IntentService/android.app.Service + */ + val ServiceMethods = listOf( + "void onCreate()", + "void onStart(android.content.Intent,int)", + "int onStartCommand(android.content.Intent,int,int)", + "android.os.IBinder onBind(android.content.Intent)", + "void onRebind(android.content.Intent)", + "boolean onUnbind(android.content.Intent)", + "void onDestroy()", + ) + + /** + * all the overridable methods of android.content.BroadcastReceiver + */ + val BroadcastReceiverMethods = listOf( + "void onReceive(android.content.Context,android.content.Intent)", + ) + + /** + * all the overridable methods of android.content.ContentProvider + */ + val ProviderMethods = listOf( + "boolean onCreate()", + ) + val FragmentMethods = listOf( + "android.view.View onCreateView(android.view.LayoutInflater,android.view.ViewGroup,android.os.Bundle)", + "void onAttach(android.content.Context)", + "void onAttach(android.app.Activity)", + "void onCreate(android.os.Bundle)", + "void onViewCreated(android.view.View,android.os.Bundle)", + "void onStart()", + "void onResume()", + "void onPause()", + "void onStop()", + "void onDestroyView()", + "void onDestroy()", + "void onDetach()", + "void onActivityCreated(android.os.Bundle)", + "void onActivityResult(int,int,android.content.Intent)", + "void onAttachFragment(androidx.fragment.app.Fragment)", + ) + + /** + * return true when sc is an android component class + */ + fun isComponentClass(sc: SootClass): Boolean { + val classes = mutableListOf( + ActivityClass, + ServiceClass, + BroadcastReceiverClass, + BroadcastReceiverClass, + FragmentClass, + ContentProviderClass + ) + if (FragmentClass != null) { + classes.add(FragmentClass) + } + classes.any { + it != null && isSubClass(sc, it) + } + return false + } + + /** + * returns true if ancestor has an ancestor relationship with child, false otherwise + */ + fun isSubClass(child: SootClass, ancestor: SootClass): Boolean { + if (child == ancestor) { + return true + } + if (child.hasSuperclass()) { + return isSubClass(child.superclass, ancestor) + } + return false + } + +} diff --git a/src/main/kotlin/net/bytedance/security/app/engineconfig/CallbackConfig.kt b/src/main/kotlin/net/bytedance/security/app/engineconfig/CallbackConfig.kt new file mode 100644 index 0000000..d6c2139 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/engineconfig/CallbackConfig.kt @@ -0,0 +1,121 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.engineconfig + +import net.bytedance.security.app.Log +import soot.Scene +import soot.SootClass +import java.util.concurrent.atomic.AtomicBoolean + + +/** + * callback config in memory + */ +class CallbackConfig(val callbackData: CallbackData) { + private var param: Map> = HashMap() + var enhanceIgnore: List = ArrayList() + private var isInit = AtomicBoolean(false) + + init { + this.enhanceIgnore = callbackData.enhanceIgnore + } + + @Synchronized + fun getCallBackConfig(): Map> { + if (!isInit.get()) { + loadConfig() + } + return param + } + + @Synchronized + private fun loadConfig() { + //double check + if (isInit.get()) { + return + } + if (Scene.v().classes.size == 0) { + throw Exception("soot not init") + } + parseRules() + isInit.set(true) + } + + private fun parseRules() { + param = getOneItem(callbackData.param) + } + + private fun getOneItem( + param: Map>, + ): MutableMap> { + val classMap = HashMap>() + // "android.view.View$OnClickListener":["*"], + // "java.lang.Runnable":["void run()"], + for ((className, methodArgArray) in param) { + val sc = Scene.v().getSootClassUnsafe(className, false) ?: continue + + val methodList: MutableList = ArrayList() + if (methodArgArray.size == 1 && methodArgArray[0] == "*") { + for (sootMethod in sc.methods) { + if (sootMethod.isConstructor || sootMethod.isStaticInitializer) { + continue + } + methodList.add(sootMethod.subSignature) + } + } else { + for (method in methodArgArray) { + methodList.add(method) + } + } + classMap[sc] = methodList + val subClasses = HashSet() + getSubCLassExcludeLib(sc, subClasses) + if (subClasses.isEmpty()) { + continue + } + for (subClass in subClasses) { + val list = classMap.computeIfAbsent(subClass) { ArrayList() } + list.addAll(methodList) + } + } + Log.logDebug("Expand param callback rules " + classMap.keys.size) + return classMap + } + + fun getSubCLassExcludeLib(sc: SootClass, subClasses: MutableSet) { + val subClassSet = if (sc.isInterface) { + Scene.v().orMakeFastHierarchy.getAllImplementersOfInterface(sc) + + } else { + Scene.v().orMakeFastHierarchy.getSubclassesOf(sc) + } + if (subClassSet == null) { + return + } + for (sootClass in subClassSet) { + if (subClasses.contains(sootClass)) { + continue + } + val className = sootClass.name + if (!EngineConfig.libraryConfig.isLibraryClass(className)) { + subClasses.add(sootClass) + } + getSubCLassExcludeLib(sootClass, subClasses) + } + } +} diff --git a/src/main/kotlin/net/bytedance/security/app/engineconfig/EngineConfig.kt b/src/main/kotlin/net/bytedance/security/app/engineconfig/EngineConfig.kt new file mode 100644 index 0000000..45ba165 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/engineconfig/EngineConfig.kt @@ -0,0 +1,107 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.engineconfig + +import kotlinx.serialization.SerialName +import kotlinx.serialization.Serializable +import net.bytedance.security.app.Log +import net.bytedance.security.app.getConfig +import net.bytedance.security.app.util.Json +import java.io.IOException +import java.nio.file.Files +import java.nio.file.Paths + + +fun isLibraryClass(className: String): Boolean { + return EngineConfig.libraryConfig.isLibraryClass(className) +} + +@Serializable +data class LibraryData(var Package: List, val ExcludeLibraryContains: List) + +@Serializable +data class CallbackData(val param: Map>, val enhanceIgnore: List) + +@Serializable +data class IgnoreListsData( + val PackageName: List? = null, + val MethodName: List? = null, + //MethodSignature differs from MethodName in that the former is a complete function signature, + val MethodSignature: List? = null +) + +@Serializable +data class FlowRuleData( + val MethodName: Map>? = null, + val MethodSignature: Map>? = null, +) + +@Serializable +data class VariableFlowRuleData( + val InstantDefault: Map? = null, + val InstantSelfDefault: Map? = null, + val StaticDefault: Map? = null, + val MethodName: Map>? = null, + val MethodSignature: Map>? = null, +) + +@Serializable +data class EngineConfigData( + @SerialName("Library") + val libraryConfig: LibraryData = LibraryData(listOf(), listOf()), + @SerialName("Callback") + val callbackConfig: CallbackData = CallbackData(mapOf(), listOf()), + @SerialName("IgnoreList") + val ignoreListConfig: IgnoreListsData = IgnoreListsData(), + @SerialName("PointerFlowRule") + val PointerPropagationConfig: FlowRuleData = FlowRuleData(), + @SerialName("VariableFlowRule") + val VariableFlowConfig: VariableFlowRuleData = VariableFlowRuleData(), +) + +object EngineConfig { + val callbackConfig: CallbackConfig + val libraryConfig: LibraryConfig + val IgnoreListConfig: IgnoreListsConfig + val PointerPropagationConfig: PointerPropagationConfig + val variableFlowConfig: VariableFlowConfig + + /** + * Make sure that Config is properly initialized before accessing EngineConfig + */ + init { + val s = loadConfigOrQuit("${getConfig().configPath}/EngineConfig.json5") + val engineConfigData = Json.decodeFromString(s) + callbackConfig = CallbackConfig(engineConfigData.callbackConfig) + libraryConfig = LibraryConfig(engineConfigData.libraryConfig) + IgnoreListConfig = IgnoreListsConfig(engineConfigData.ignoreListConfig) + PointerPropagationConfig = PointerPropagationConfig(engineConfigData.PointerPropagationConfig) + variableFlowConfig = VariableFlowConfig(engineConfigData.VariableFlowConfig) + } + + fun loadConfigOrQuit(path: String): String { + Log.logInfo("Load config file $path") + val jsonStr = try { + String(Files.readAllBytes(Paths.get(path))) + } catch (e: IOException) { + Log.logErr("read config file $path failed") + throw Exception("read config file $path failed") + } + return jsonStr + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/engineconfig/IgnoreListsConfig.kt b/src/main/kotlin/net/bytedance/security/app/engineconfig/IgnoreListsConfig.kt new file mode 100644 index 0000000..0b8bd49 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/engineconfig/IgnoreListsConfig.kt @@ -0,0 +1,65 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.engineconfig + + +/** +functions such as hot patches, do not need to be analyzed. + */ +class IgnoreListsConfig(ignoreListData: IgnoreListsData) { + + private var packageNameSet: HashSet = HashSet() + private var methodNameSet: HashSet = HashSet() + private var methodSigSet: HashSet = HashSet() + + init { + ignoreListData.PackageName?.forEach { + packageNameSet.add(it) + } + ignoreListData.MethodName?.forEach { + methodNameSet.add(it) + } + ignoreListData.MethodSignature?.forEach { methodSigSet.add(it) } + + } + + fun isInIgnoreList(className: String, methodName: String, methodSig: String): Boolean { + return containsPackageName(className) || containsMethodName(methodName) || containsMethodSig(methodSig) + } + + private fun containsPackageName(className: String): Boolean { + if (packageNameSet.contains(className)) { + return true + } + for (ignorePackageName in packageNameSet) { + if (className.startsWith(ignorePackageName)) { + return true + } + } + return false + } + + private fun containsMethodName(methodName: String): Boolean { + return methodNameSet.contains(methodName) + } + + private fun containsMethodSig(methodSig: String): Boolean { + return methodSigSet.contains(methodSig) + } + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/engineconfig/LibraryConfig.kt b/src/main/kotlin/net/bytedance/security/app/engineconfig/LibraryConfig.kt new file mode 100644 index 0000000..27b0610 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/engineconfig/LibraryConfig.kt @@ -0,0 +1,54 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.engineconfig + +class LibraryConfig(val libraryData: LibraryData) { + + private fun isInExcludeLibrary(className: String): Boolean { + for (excludeContain in libraryData.ExcludeLibraryContains) { + if (className.contains(excludeContain)) { + return true + } + } + return false + } + + fun isLibraryClass(className: String): Boolean { + for (packageName in libraryData.Package) { + //If it's library. It is necessary to continue to check + // whether it belongs to the whitelist which needs to be analyzed + if (className.startsWith(packageName)) { + // Only those not on the whitelist are considered libraries + if (!isInExcludeLibrary(className)) { + return true + } + } + } + return false + } + + fun isLibraryMethod(methodSig: String): Boolean { + val className = methodSig.substring(1) + return isLibraryClass(className) + } + + fun setPackage(packages: List) { + libraryData.Package = packages + } + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/engineconfig/PointerPropagationConfig.kt b/src/main/kotlin/net/bytedance/security/app/engineconfig/PointerPropagationConfig.kt new file mode 100644 index 0000000..30faf70 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/engineconfig/PointerPropagationConfig.kt @@ -0,0 +1,68 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +@file:Suppress("unused") + +package net.bytedance.security.app.engineconfig + +import kotlinx.serialization.Serializable +import net.bytedance.security.app.TaintTweakData + +@Serializable +data class IOData(val I: List, val O: List) +data class Propagation(val from: String, val to: String) + + +class PointerPropagationConfig { + + //key method, value=>{pointer from Pair.first propagate to Pair.second} + var methodSigRule: Map> = mapOf() + var methodNameRule: Map> = mapOf() + + constructor(flowRuleData: FlowRuleData) { + if (flowRuleData.MethodName != null) { + this.methodNameRule = VariableFlowConfig.parseMapRule(flowRuleData.MethodName) + } + if (flowRuleData.MethodSignature != null) { + this.methodSigRule = VariableFlowConfig.parseMapRule(flowRuleData.MethodSignature) + } + } + + constructor(defaultConfig: PointerPropagationConfig, wrapperData: TaintTweakData) { + if (wrapperData.MethodName != null) { + this.methodNameRule = VariableFlowConfig.parseMapRule(wrapperData.MethodName) + } + if (wrapperData.MethodSignature != null) { + this.methodSigRule = VariableFlowConfig.parseMapRule(wrapperData.MethodSignature) + } + if (wrapperData.DisableEngineWrapper == true) { + return + } + val methodNameRule = HashMap(defaultConfig.methodNameRule) + val methodSigRule = HashMap(defaultConfig.methodSigRule) + this.methodNameRule.forEach { (k, v) -> + methodNameRule[k] = v + } + this.methodSigRule.forEach { (k, v) -> + methodSigRule[k] = v + } + this.methodNameRule = methodNameRule + this.methodSigRule = methodSigRule + } + + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/engineconfig/VariableFlowConfig.kt b/src/main/kotlin/net/bytedance/security/app/engineconfig/VariableFlowConfig.kt new file mode 100644 index 0000000..66e21a7 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/engineconfig/VariableFlowConfig.kt @@ -0,0 +1,120 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +@file:Suppress("unused") + +package net.bytedance.security.app.engineconfig + +import net.bytedance.security.app.TaintTweakData + + +class VariableFlowConfig { + // taint flow forInstant + var instantDefaultRule: List = listOf() + + // taint flow for callsite where caller and callee are in the same class + var instantSelfDefaultRule: List = listOf() + + // taint flow for static method + var staticDefaultRule: List = listOf() + var methodSigRule: Map> = mapOf() + var methodNameRule: Map> = mapOf() + + constructor(wrapperData: VariableFlowRuleData) { + if (wrapperData.InstantDefault != null) { + this.instantDefaultRule = parseListRule(wrapperData.InstantDefault) + } + if (wrapperData.InstantSelfDefault != null) { + this.instantSelfDefaultRule = parseListRule(wrapperData.InstantSelfDefault) + } + if (wrapperData.StaticDefault != null) { + this.staticDefaultRule = parseListRule(wrapperData.StaticDefault) + } + if (wrapperData.MethodName != null) { + this.methodNameRule = parseMapRule(wrapperData.MethodName) + } + if (wrapperData.MethodSignature != null) { + this.methodSigRule = parseMapRule(wrapperData.MethodSignature) + } + } + + constructor(defaultCfg: VariableFlowConfig, wrapperData: TaintTweakData) { + this.instantDefaultRule = defaultCfg.instantDefaultRule + this.instantSelfDefaultRule = defaultCfg.instantSelfDefaultRule + this.staticDefaultRule = defaultCfg.staticDefaultRule + if (wrapperData.MethodName != null) { + this.methodNameRule = parseMapRule(wrapperData.MethodName) + } + if (wrapperData.MethodSignature != null) { + this.methodSigRule = parseMapRule(wrapperData.MethodSignature) + } + + if (wrapperData.DisableEngineWrapper == true) { + return + } + val methodNameRule = HashMap(defaultCfg.methodNameRule) + val methodSigRule = HashMap(defaultCfg.methodSigRule) + this.methodNameRule.forEach { (k, v) -> + methodNameRule[k] = v + } + this.methodSigRule.forEach { (k, v) -> + methodSigRule[k] = v + } + this.methodNameRule = methodNameRule + this.methodSigRule = methodSigRule + } + + companion object { + fun parseListRule(jsonObject: Map): List { + val r = ArrayList() + for (ruleObj in jsonObject.values) { + val inArr = ruleObj.I + val outArr = ruleObj.O + for (`in` in inArr) { + for (out in outArr) { + r.add(Propagation(`in`, out)) + } + } + } + return r + } + + + fun parseMapRule(jsonObject: Map>): HashMap> { + val r = HashMap>() + for ((methodName, methodRule) in jsonObject) { + var ruleList = r[methodName] + if (ruleList == null) { + ruleList = ArrayList() + r[methodName] = ruleList + } + ruleList.clear() + for (ruleObj in methodRule.values) { + val inArr = ruleObj.I + val outArr = ruleObj.O + for (`in` in inArr) { + for (out in outArr) { + ruleList.add(Propagation(`in`, out)) + } + } + } + } + return r + } + } + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/pathfinder/PathResult.kt b/src/main/kotlin/net/bytedance/security/app/pathfinder/PathResult.kt new file mode 100644 index 0000000..3e807e5 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/pathfinder/PathResult.kt @@ -0,0 +1,33 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.pathfinder + +import net.bytedance.security.app.pointer.PLPointer +import soot.SootMethod +import soot.jimple.Stmt + +/** + * there is a taint flow on the `stmt` of `method` + */ +class TaintEdge(val method: SootMethod, val stmt: Stmt) { + override fun toString(): String { + return stmt.toString() + } +} + +data class PathResult(val curPath: List) \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/pathfinder/TaintFlowEdgeFinder.kt b/src/main/kotlin/net/bytedance/security/app/pathfinder/TaintFlowEdgeFinder.kt new file mode 100644 index 0000000..e996a0d --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/pathfinder/TaintFlowEdgeFinder.kt @@ -0,0 +1,303 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.pathfinder + +import net.bytedance.security.app.PLUtils +import net.bytedance.security.app.pointer.PLLocalPointer +import net.bytedance.security.app.pointer.PLPointer +import net.bytedance.security.app.pointer.PLPtrObjectField +import net.bytedance.security.app.pointer.PLPtrStaticField +import net.bytedance.security.app.rules.TaintFlowRule +import soot.Scene +import soot.SootField +import soot.UnknownType +import soot.Value +import soot.jimple.Constant +import soot.jimple.FieldRef +import soot.jimple.Stmt +import soot.jimple.internal.JAssignStmt +import soot.jimple.internal.JReturnStmt +import soot.jimple.internal.JimpleLocal + +/** +give the source and destination pointer, find the jimple statements +that assign the source pointer to the destination pointer + */ +class TaintFlowEdgeFinder(val rule: TaintFlowRule) { + + companion object { + /** + * give the source and destination pointer, find the jimple statements + * that assign the source pointer to the destination pointer + * @return for dst=obj.f() ,src is in the method f,and dst is in the caller of method f, + * so there are two statements connect the src and dst + */ + fun getPossibleEdge(srcPtr: PLPointer, dstPtr: PLPointer): List? { + if (srcPtr is PLLocalPointer) { + if (dstPtr is PLLocalPointer) { + if (dstPtr.isThis) { + return localToThis(srcPtr, dstPtr) + } else if (dstPtr.isParam) { + return localToParam(srcPtr, dstPtr) + } else { + return localToLocal(srcPtr, dstPtr) + } + } else if (dstPtr is PLPtrStaticField) { + val field = Scene.v().getField(dstPtr.signature()) + return localToField(srcPtr, field) + } else { // PLPtrObjectField + return localToObjField(srcPtr, dstPtr as PLPtrObjectField) + } + } else if (srcPtr is PLPtrStaticField) { + val field = Scene.v().getField(srcPtr.signature()) + return fieldToLocal(field, dstPtr as PLLocalPointer) + } else { // PLPtrObjectField + return objFieldToLocal(srcPtr as PLPtrObjectField, dstPtr as PLLocalPointer) + } + } + + /* + * obj.f(arg1) + * flow from arg1 to obj + * + * */ + private fun localToThis(localPtr: PLLocalPointer, thisPtr: PLLocalPointer): List? { + val edges = ArrayList() + //they must be in the same method + assert(localPtr.method != thisPtr.method) + for (unit in localPtr.method.activeBody.units) { + val stmt = unit as Stmt + if (!stmt.containsInvokeExpr()) { + continue + } + if (stmt.invokeExpr.method != thisPtr.method) { + continue + } + for (valueBox in stmt.useAndDefBoxes) { + val v = valueBox.value + if (v !is JimpleLocal || v.name != localPtr.variable) { + continue + } + edges.add(TaintEdge(localPtr.method, stmt)) + } + } + if (edges.isEmpty()) { + return null + } + return edges + } + + /** + * + * obj.f(r1) + * flow from r1 to @parameter0 + */ + private fun localToParam(srcPtr: PLLocalPointer, dstPtr: PLLocalPointer): List? { + val edges = ArrayList() + assert(dstPtr.isParam) + for (unit in srcPtr.method.activeBody.units) { + val stmt = unit as Stmt + if (!stmt.containsInvokeExpr()) { + continue + } + if (stmt.invokeExpr.method != dstPtr.method) { + continue + } + for (valueBox in stmt.useAndDefBoxes) { + val v = valueBox.value + if (srcPtr.isConstStr) { + if (v !is Constant || PLUtils.constSig(v) != srcPtr.variable) { + continue + } + } else { + if (v !is JimpleLocal || v.name != srcPtr.variable) { + continue + } + } + edges.add(TaintEdge(srcPtr.method, stmt)) + } + } + if (edges.isEmpty()) { + return null + } + + return edges + } + + /** + there are three types: + 1. dst=src+r3 + 2. dst=obj.f(src) through user specified rule ,but not analyze + 3. dst=obj.f() through return of f + */ + private fun localToLocal(srcPtr: PLLocalPointer, dstPtr: PLLocalPointer): List? { + assert(dstPtr.isLocal) + if (srcPtr.method == dstPtr.method) { + return localToLocalInOneMethod(srcPtr, dstPtr) + } else { + return localToLocalThroughReturn(srcPtr, dstPtr) + } + } + + /** + * 1. dst=src+r3 + * 2. dst=obj.f(src) + */ + private fun localToLocalInOneMethod(srcPtr: PLLocalPointer, dstPtr: PLLocalPointer): List? { + val edges = ArrayList() + for (unit in srcPtr.method.activeBody.units) { + val stmt = unit as Stmt + var hasSrc = false + var hasDst = false + for (valueBox in stmt.useAndDefBoxes) { + val v: Value = valueBox.value + if (v.variableName() == srcPtr.variableName) + hasSrc = true + if (v.variableName() == dstPtr.variableName) + hasDst = true + } + if (hasSrc && hasDst) { + edges.add(TaintEdge(srcPtr.method, stmt)) + } + } + if (edges.isEmpty()) { + return null + } + + return edges + } + + /** + * dst=obj.f() + */ + private fun localToLocalThroughReturn(srcPtr: PLLocalPointer, dstPtr: PLLocalPointer): List? { + val edges = ArrayList() + assert(dstPtr.isLocal && srcPtr.isLocal && srcPtr.method != dstPtr.method) + for (unit in srcPtr.method.activeBody.units) { + val stmt = unit as Stmt + if (stmt !is JReturnStmt) { + continue + } + for (valueBox in stmt.useAndDefBoxes) { + val v = valueBox.value + if (v !is JimpleLocal || v.name != srcPtr.variable) { + continue + } + edges.add(TaintEdge(srcPtr.method, stmt)) + } + } + for (unit in dstPtr.method.activeBody.units) { + val stmt = unit as Stmt + if (!stmt.containsInvokeExpr()) { + continue + } + if (stmt.invokeExpr.method != srcPtr.method) { + continue + } + for (valueBox in stmt.useAndDefBoxes) { + val v = valueBox.value + if (v !is JimpleLocal || v.name != dstPtr.variable) { + continue + } + edges.add(TaintEdge(dstPtr.method, stmt)) + } + } + if (edges.isEmpty()) { + return null + } + return edges + } + + //r0.field=r1 + private fun localToField( + srcPtr: PLLocalPointer, + dst: SootField, + ): List? { + val edges = ArrayList() + for (unit in srcPtr.method.activeBody.units) { + val stmt = unit as Stmt + if (stmt !is JAssignStmt) { + continue + } + val leftExpr = stmt.leftOp + val rightExpr = stmt.rightOp + if (rightExpr !is JimpleLocal || rightExpr.name != srcPtr.variable) { + continue + } + if (leftExpr !is FieldRef || leftExpr.field != dst) { + continue + } + edges.add(TaintEdge(srcPtr.method, stmt)) + } + if (edges.isEmpty()) { + return null + } + return edges + } + + private fun localToObjField(srcPtr: PLLocalPointer, dstPtr: PLPtrObjectField): List? { + if (dstPtr.ptrType !is UnknownType) { + return localToField(srcPtr, dstPtr.sootField!!) + } + return null + } + + //r1=r0.field + private fun fieldToLocal( + src: SootField, + dstPtr: PLLocalPointer + ): List? { + val edges = ArrayList() + for (unit in dstPtr.method.activeBody.units) { + val stmt = unit as Stmt + if (stmt !is JAssignStmt) { + continue + } + val leftExpr = stmt.leftOp + val rightExpr = stmt.rightOp + if (leftExpr !is JimpleLocal || leftExpr.name != dstPtr.variable) { + continue + } + if (rightExpr !is FieldRef || rightExpr.field != src) { + continue + } + edges.add(TaintEdge(dstPtr.method, stmt)) + } + if (edges.isEmpty()) { + return null + } + return edges + } + + private fun objFieldToLocal(srcPtr: PLPtrObjectField, dstPtr: PLLocalPointer): List? { + if (srcPtr.sootField != null) { + return fieldToLocal(srcPtr.sootField!!, dstPtr) + } + return null + } + } +} + +fun Value.variableName(): String { + val s = this.toString() + val i = s.indexOf(":") + if (i < 0) { + return s + } + return s.slice(0 until i) +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/pathfinder/TaintPathFinder.kt b/src/main/kotlin/net/bytedance/security/app/pathfinder/TaintPathFinder.kt new file mode 100644 index 0000000..ac3e38f --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/pathfinder/TaintPathFinder.kt @@ -0,0 +1,384 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.pathfinder + +import net.bytedance.security.app.Log +import net.bytedance.security.app.PLUtils +import net.bytedance.security.app.PreAnalyzeContext +import net.bytedance.security.app.getConfig +import net.bytedance.security.app.pointer.PLLocalPointer +import net.bytedance.security.app.pointer.PLPointer +import net.bytedance.security.app.result.OutputSecResults +import net.bytedance.security.app.ruleprocessor.ConstModeProcessor +import net.bytedance.security.app.rules.ConstNumberModeRule +import net.bytedance.security.app.rules.ConstStringModeRule +import net.bytedance.security.app.rules.TaintFlowRule +import net.bytedance.security.app.sanitizer.SanitizeContext +import net.bytedance.security.app.sanitizer.SanitizerFactory +import net.bytedance.security.app.taintflow.AnalyzeContext +import net.bytedance.security.app.taintflow.TaintAnalyzer +import net.bytedance.security.app.taintflow.TwoStagePointerAnalyze +import net.bytedance.security.app.ui.TaintPathModeHtmlWriter +import net.bytedance.security.app.util.toFormatedString +import net.bytedance.security.app.util.toSortedMap +import net.bytedance.security.app.util.toSortedSet +import soot.jimple.IntConstant +import soot.jimple.LongConstant +import soot.jimple.NumericConstant +import soot.jimple.StringConstant +import java.util.* + +/** + * After pointer analyze, find the shortest path from source to sink, + * and then generate the final vulnerability report + */ +class TaintPathFinder( + val ctx: PreAnalyzeContext, + private val analyzeContext: AnalyzeContext, + val rule: TaintFlowRule, + val analyzer: TaintAnalyzer, +) { + //todo Remember that PLPointer in Analyzer and Sanitizer are different from PLPointer in PointFactory + suspend fun analyze() { + Log.logDebug(String.format("start analyze pointers %s", rule.name)) + if (rule is ConstStringModeRule || rule is ConstNumberModeRule) { + constAnalyze() + } else { + taintAnalyze() + } + } + + /** + *ConstNumberMode and ConstStringMode don't support sanitizer + */ + private suspend fun constAnalyze() { + runConstAnalyze(analyzer) + } + + private suspend fun taintAnalyze() { + runTaintAnalyze(analyzer) + } + + /** + * find the path from the const string or number to the sink, + * 1. find all pointers can flow to sink + * 2. filter these pointers if they are const string or number + * 3. write the path from each const to the sink to the result + * */ + private suspend fun runConstAnalyze(analyzer: TaintAnalyzer) { + for (ptr in analyzer.sinkPtrSet) { + val sinkTaintedSet = analyzeContext.collectReversePropagation(ptr, rule.primTypeAsTaint) + if (sinkTaintedSet.isEmpty()) { + Log.logDebug("nodes empty $ptr") + continue + } + val strSet: MutableSet = HashSet() + val numSet: MutableSet> = HashSet() + for (ptrLocal in sinkTaintedSet) { + if (ptrLocal !is PLLocalPointer || ptrLocal.constant == null) { + continue + } + val constant = ptrLocal.constant + if (constant is StringConstant) { + strSet.add(ptrLocal) + } else if (constant is NumericConstant) { + if (constant is IntConstant) { + numSet.add(Pair(ptrLocal, constant.value.toLong())) + } else if (constant is LongConstant) { + numSet.add(Pair(ptrLocal, constant.value)) + } + } + } + if (rule is ConstStringModeRule) { + for (constStr in strSet) { + Log.logDebug("const str $constStr") + findConstStringPath( + constStr, + analyzer.sinkPtrSet + ) + } + } else if (rule is ConstNumberModeRule) { + for (num in numSet) { + Log.logDebug("const num $num") + findConstNumberPath( + num, + analyzer.sinkPtrSet + ) + } + } + } + } + + /** + * if the src satisfy the constraints ,then find the shortest path. + * exists of the path is verified + */ + private suspend fun findConstNumberPath( + src: Pair, + sinkPtrSet: Set, + ) { + val isMatch = isConstNumRuleMatch(src.second) + if (!isMatch) { + return + } + findConstPath(src.first, sinkPtrSet) + return + } + + /** + * constraints for ConstNumberMode + */ + private fun isConstNumRuleMatch(constNumber: Long): Boolean { + assert(rule is ConstNumberModeRule) + val constNumberModeRule = rule as ConstNumberModeRule + if (constNumberModeRule.targetNumberArr == null) { + return true + } + if (constNumberModeRule.targetNumberArr.contains(constNumber.toInt())) { + return true + } + return false + } + + /** + * constraints for ConstStringMode + */ + private fun isConstStringRuleMatch(constStr: String): Boolean { + var isMatch = false + if (rule !is ConstStringModeRule) { + return false + } + if (rule.constLen != null) { + val constStrLen = constStr.length + if (constStrLen > 0 && constStrLen % rule.constLen == 0) { + if (rule.targetStringArr != null) { + if (ConstModeProcessor.isMatchTargetConstStr(constStr, rule.targetStringArr)) { + isMatch = true + } + } else { + isMatch = true + } + } + } else { + if (rule.targetStringArr != null) { + if (ConstModeProcessor.isMatchTargetConstStr(constStr, rule.targetStringArr)) { + isMatch = true + } + } else if (rule.minLen != null) { + if (constStr.length >= rule.minLen) { + isMatch = true + } + } else { + isMatch = true + } + } + return isMatch + } + + + private suspend fun findConstPath( + srcPtr: PLLocalPointer, + sinkPtrSet: Set, + ) { + calcPath(srcPtr, sinkPtrSet) + } + + /** + * if the src satisfy the constraints ,then find the shortest path. + * exists of the path is verified + */ + private suspend fun findConstStringPath( + srcPtr: PLLocalPointer, + sinkPtrSet: Set + ) { + val constStr = srcPtr.variableName + val isMatch = isConstStringRuleMatch(constStr) + if (!isMatch) { + return + } + findConstPath(srcPtr, sinkPtrSet) + return + } + + private suspend fun runTaintAnalyze(analyzer: TaintAnalyzer) { + calcPathFromSourceTaint( + analyzer.sourcePtrSet, + analyzer.sinkPtrSet + ) + + } + + /** + * filter by sanitizer then find the shortest path. + * @param sourcePtrSet + * @param sinkPtrSet + * todo use multi threads to search + */ + private suspend fun calcPathFromSourceTaint( + sourcePtrSet: Set, + sinkPtrSet: Set, + ) { + for (sourcePtr in sourcePtrSet) { + if (checkSanitizeRules(sourcePtr)) { + Log.logDebug("Sanitize Check Pass") + continue + } + Log.logDebug("======> ParamSources Calculate realizable path from $sourcePtr") + TwoStagePointerAnalyze.recordMethodTakesTime( + "calcPathFromSourceTaint ${this.analyzer.entryMethod.signature}-${sourcePtr.signature()}", + 3000 + ) { + calcPath( + sourcePtr, + sinkPtrSet, + ) + } + } + } + + private fun isThisSolverNeedLog(): Boolean { + return this.rule.isThisRuleNeedLog() + } + + /** + * Try to find the shortest path from srcPtr to sinkPtrSet + */ + private suspend fun calcPath( + srcPtr: PLPointer, + sinkPtrSet: Set, + ) { + if (isThisSolverNeedLog()) { + val sb = StringBuilder() + val sinkTaintedSet = HashSet() + for (sink in sinkPtrSet) { + sinkTaintedSet.addAll(analyzeContext.collectReversePropagation(sink, rule.primTypeAsTaint)) + } + sb.append("sinkPtrSet=${sinkPtrSet.toSortedSet()}, taint sinkNodeSet: ${sinkTaintedSet.toSortedSet()}\n") + PLUtils.writeFile(getConfig().outPath + "/sink.log", sb.toString()) + sb.clear() + sb.append("\n\n\n\n\n\nsrcPtr=${srcPtr}, taint sourceNodeSet:\n") + val srcTaintedSet = analyzeContext.collectPropagation(srcPtr, rule.primTypeAsTaint) + sb.append("\n\nsrcTaintedSet=${srcTaintedSet.toSortedSet()}") + PLUtils.writeFile(getConfig().outPath + "/source.log", sb.toString()) + sb.clear() + PLUtils.writeFile( + getConfig().outPath + "/ptrToSet.log", + analyzeContext.pointerToObjectSet.toSortedMap().toFormatedString() + ) + PLUtils.writeFile( + getConfig().outPath + "/taintPtrFlowGraph.log", + analyzeContext.variableFlowGraph.toSortedMap().toFormatedString() + ) + PLUtils.writeFile( + getConfig().outPath + "/ptrFlowGraph.log", + analyzeContext.pointerFlowGraph.toSortedMap().toFormatedString() + ) + PLUtils.writeFile(getConfig().outPath + "rm.log", analyzeContext.rm.toSortedSet().toFormatedString()) +// exitProcess(3) + } + val g = analyzeContext.variableFlowGraph + val path = bfsSearch(srcPtr, sinkPtrSet, g, getConfig().maxPathLength, rule.name) ?: return + val result = PathResult(path) + try { + TaintPathModeHtmlWriter(OutputSecResults, analyzer, result, rule).addVulnerabilityAndSaveResultToOutput() + } catch (ex: Exception) { + ex.printStackTrace() + } + } + + + /** + * entry point of the sanitizer + */ + private fun checkSanitizeRules(sourcePtr: PLLocalPointer): Boolean { + val sanitizeContext = SanitizeContext(analyzeContext, sourcePtr) + SanitizerFactory.createSanitizers(rule, ctx).forEach { + if (it.matched(sanitizeContext)) { + return true + } + } + return false + } + + + companion object { + data class PointerAndDepth(val p: PLPointer, val depth: Int) + + /** + * breath first search for the shortest path + * @param sourcePointer source + * @param destinations possible destinations + * @param g graph + * @param maxPath the maximum size of the shortest path + * @param name for log + * @return the shortest path from src to dst or null if no path found + */ + fun bfsSearch( + sourcePointer: PLPointer, + destinations: Set, + g: Map>, + maxPath: Int, + name: String, + ): List? { + val parents = HashMap() + val queue = LinkedList() + val visited = HashSet() + queue.addLast(PointerAndDepth(sourcePointer, 0)) + var validDst: PLPointer? = null + visited.add(sourcePointer) + found@ while (queue.isNotEmpty()) { + val src = queue.pollFirst() + if (destinations.contains(src.p)) { + validDst = src.p + break@found + } + + if (src.depth >= maxPath) { + break //The path length exceeded the upper limit + } + val next = g[src.p] ?: continue + + for (n in next) { + if (visited.contains(n)) { + continue + } + visited.add(n) + parents[n] = src.p + queue.addLast(PointerAndDepth(n, src.depth + 1)) + } + } + if (validDst == null) { + return null + } + + val path = ArrayList() + var n = validDst + while (n != null) { + path.add(n) + n = parents[n] + } + if (path.size >= maxPath) { + Log.logWarn("path too long: for $name is discarded,currentLen=${path.size},maxLen=$maxPath") + return null + } + return path.reversed() + } + + + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/pointer/PLLocalPointer.kt b/src/main/kotlin/net/bytedance/security/app/pointer/PLLocalPointer.kt new file mode 100644 index 0000000..71a6e38 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/pointer/PLLocalPointer.kt @@ -0,0 +1,139 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.pointer + +import net.bytedance.security.app.PLUtils +import net.bytedance.security.app.util.profiler +import soot.SootField +import soot.SootMethod +import soot.Type +import soot.jimple.Constant + +/** + * to save memory + */ +const val shortNameEnable = true + +/** + * Pointer to variables and constants generated during analysis + */ +class PLLocalPointer : PLPointer { + var method: SootMethod + var variable: String + var id: String + + override val ptrType: Type + var constant: Constant? = null + fun setConst(constant: Constant?) { + this.constant = constant + } + + constructor(method: SootMethod, localName: String, origType: Type, sig: String) { + this.method = method + variable = localName + + ptrType = PointerFactory.typeWrapper(origType) + + id = sig + profiler.newPtrLocal(id) + } + + constructor(method: SootMethod, localName: String, origType: Type) { + this.method = method + variable = localName + // We have a rule that all arrays are converted to 1-dimensional arrays. + ptrType = PointerFactory.typeWrapper(origType) + id = getPointerLocalSignature(method, localName) + profiler.newPtrLocal(id) + } + + val isParam: Boolean + get() = variable.startsWith(PLUtils.PARAM) + + val isConstStr: Boolean + get() = variable.startsWith(PLUtils.CONST_STR) + + val isThis: Boolean + get() = variable == PLUtils.THIS_FIELD + val isLocal: Boolean + get() = !isParam && !isThis + + /** + * name of this variable,for example r0,$r0, + * if PLPtrLocal is a constant,then it's the value of the constant + */ + val variableName: String + get() { + if (!isConstStr) { + return variable + } + return variable.slice(PLUtils.CONST_STR.length until variable.length) + } + + override fun toString(): String { + return this.signature() + } + + override fun equals(other: Any?): Boolean { + return if (other is PLLocalPointer) { + id == other.id + } else false + } + + override fun hashCode(): Int { + return id.hashCode() + } + + override fun signature(): String { + return getLocalLongSignature(method, variable) + } + + companion object { + fun getLocalLongSignature(method: SootMethod, localName: String): String { + return "${method.signature}->$localName" + } + + fun getPointerLocalSignature(method: SootMethod, localName: String): String { + if (shortNameEnable) { + return "${method.shortSignature()}->$localName" + } + return getLocalLongSignature(method, localName) + } + } +} + +fun SootMethod.shortSignature(): String { + if (shortNameEnable) { + return "${this.declaringClass.shortName}:${this.name}.${this.number}" + } + return this.signature +} + +fun SootField.shortSignature(): String { + if (shortNameEnable) { + return "${this.declaringClass.shortName}:${this.name}.${this.number}" + } + return this.signature +} + +fun Type.shortName(): String { + if (shortNameEnable) { + return this.toString().split(".").last() + this.number.toString() + } + return this.toString() +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/pointer/PLObject.kt b/src/main/kotlin/net/bytedance/security/app/pointer/PLObject.kt new file mode 100644 index 0000000..0cfc662 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/pointer/PLObject.kt @@ -0,0 +1,98 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.pointer + +import net.bytedance.security.app.taintflow.TwoStagePointerAnalyze + +import net.bytedance.security.app.util.profiler +import soot.SootField +import soot.SootMethod +import soot.Type +import soot.Value + + +/** + * Corresponds to object in pointer analyze. For example, File file=new File() + */ +class PLObject(var classType: Type, private val where: Any, private val site: Int, val signature: String) { + val isPseudoObj: Boolean + get() = where is SootMethod && where == TwoStagePointerAnalyze.getPseudoEntryMethod() + + init { + profiler.newObject(signature) + } + + override fun toString(): String { + return this.longSignature() + } + + override fun equals(other: Any?): Boolean { + return if (other is PLObject) { + signature == other.signature + } else false + } + + override fun hashCode(): Int { + return signature.hashCode() + } + + fun longSignature(): String { + val sig = when (where) { + is SootMethod -> { + where.signature + } + is SootField -> { + where.signature + } + else -> { + throw Exception("getObjectSignature unknown where $where") + } + } + return "obj{$sig:$site=>${classType}}" + } + + companion object { + + fun getObjectSignature( + classType: Type, + where: Any, //SootMethod or SootField + v: Value?, + site: Int + ): String { + var value = "" + if (v != null) { + value = v.toString() + } + val sig = when (where) { + is SootMethod -> { + where.shortSignature() + } + is SootField -> { + where.shortSignature() + } + else -> { + throw Exception("getObjectSignature unknown where $where") + } + } + if (shortNameEnable) { + return "obj{${sig}:$site=>${classType.shortName()}}" + } + return "obj{$sig:$site=>${classType.shortName()}::{$value}}" + } + } +} diff --git a/src/main/kotlin/net/bytedance/security/app/pointer/PLPointer.kt b/src/main/kotlin/net/bytedance/security/app/pointer/PLPointer.kt new file mode 100644 index 0000000..a574cdd --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/pointer/PLPointer.kt @@ -0,0 +1,31 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.pointer + +import soot.Type + +/** + * there are three types of pointer: + * 1. local variable + * 2. static field + * 3. instance field + */ +interface PLPointer { + val ptrType: Type + fun signature(): String +} diff --git a/src/main/kotlin/net/bytedance/security/app/pointer/PLPtrObjectField.kt b/src/main/kotlin/net/bytedance/security/app/pointer/PLPtrObjectField.kt new file mode 100644 index 0000000..39b1d53 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/pointer/PLPtrObjectField.kt @@ -0,0 +1,77 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.pointer + + +import net.bytedance.security.app.util.profiler +import soot.SootField +import soot.Type + +/** + * pointer points to the field of an instance + * obj is the instance. + */ +@Suppress("ConvertSecondaryConstructorToPrimary") +class PLPtrObjectField : PLPointer { + override var ptrType: Type + + + var obj: PLObject + var field: String + + /** + * there is only one case that sootField is null, the field is @data + */ + var sootField: SootField? + var id: String + + constructor( + obj: PLObject, + field: String, + type: Type, + sootField: SootField?, + signature: String, + ) { + profiler.newPtrObjectField(signature) + + this.ptrType = type + this.obj = obj + this.field = field + this.sootField = sootField + this.id = signature + } + + override fun toString(): String { + return signature() + } + + override fun equals(other: Any?): Boolean { + return if (other is PLPtrObjectField) { + id == other.id + } else false + } + + override fun hashCode(): Int { + return id.hashCode() + } + + override fun signature(): String { + return PointerFactory.getObjectFieldSignature(obj, ptrType, field) + } + +} diff --git a/src/main/kotlin/net/bytedance/security/app/pointer/PLPtrStaticField.kt b/src/main/kotlin/net/bytedance/security/app/pointer/PLPtrStaticField.kt new file mode 100644 index 0000000..f314792 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/pointer/PLPtrStaticField.kt @@ -0,0 +1,53 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.pointer + +import net.bytedance.security.app.util.profiler +import soot.SootField +import soot.Type + +/** + * pointer representation of a static field + */ +class PLPtrStaticField(val field: SootField) : + PLPointer { + override val ptrType: Type get() = this.field.type + + + override fun toString(): String { + return field.signature + } + + init { + profiler.newPtrStaticField(field.signature) + } + + override fun equals(other: Any?): Boolean { + return if (other is PLPtrStaticField) { + field == other.field + } else false + } + + override fun hashCode(): Int { + return field.hashCode() + } + + override fun signature(): String { + return field.signature + } +} diff --git a/src/main/kotlin/net/bytedance/security/app/pointer/PointerFactory.kt b/src/main/kotlin/net/bytedance/security/app/pointer/PointerFactory.kt new file mode 100644 index 0000000..03925d6 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/pointer/PointerFactory.kt @@ -0,0 +1,174 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.pointer + +import soot.* + +/** + * PLObject and PLPointer creator. + * + */ +class PointerFactory { + var ptrIndexMap: MutableMap = HashMap() + var objIndexMap: MutableMap = HashMap() + + /** + * get or create a PLObject. + * @param clsType type of this object, for example java.io.File + * @param method where create this object + * @param site method and site Identify a unique object + */ + fun allocObject( + clsType: Type, + method: SootMethod, + v: Value?, + site: Int + ): PLObject { + val classType = typeWrapper(clsType) + val objSig = PLObject.getObjectSignature(classType, method, v, site) + if (objIndexMap.containsKey(objSig)) { + return objIndexMap[objSig]!! + } + + val obj = PLObject(classType, method, site, objSig) + objIndexMap[objSig] = obj + return obj + } + + /** + * get or create a PLObject. + * $r1=System.out + * @param clsType type of this object + * @param field the global static field + * @param site, call site info + */ + fun allocObjectByStaticField( + clsType: Type, + field: SootField, + v: Value?, + site: Int + ): PLObject { + val classType = typeWrapper(clsType) + val objSig = PLObject.getObjectSignature(classType, field, v, site) + if (objIndexMap.containsKey(objSig)) { + return objIndexMap[objSig]!! + } + + val obj = PLObject(classType, field, site, objSig) + objIndexMap[objSig] = obj + return obj + } + + /** + * get or create a PLPtrLocal + * @param method which this pointer belongs to + * @param localName of this variable,for example r0,$r1, or @const_str:some_string + * @param origType type of this variable,for example java.io.File + */ + fun allocLocal( + method: SootMethod, + localName: String, + origType: Type + ): PLLocalPointer { + val ptrSig = + PLLocalPointer.getPointerLocalSignature(method, localName) + if (ptrIndexMap.containsKey(ptrSig)) { + return ptrIndexMap[ptrSig] as PLLocalPointer + } + val ptr = PLLocalPointer(method, localName, origType, ptrSig) + ptrIndexMap[ptrSig] = ptr + + return ptr + } + + //for test only + @Suppress("unused") + fun testGetLocal(methodSignature: String, localName: String): PLPointer? { + val method = Scene.v().getMethod(methodSignature) ?: return null + val ptrSig = + PLLocalPointer.getPointerLocalSignature(method, localName) + return ptrIndexMap[ptrSig] + } + + /** + * get or create a PLPtrObjectField + * @param obj object this field belongs to + * @param fieldName name of this field, maybe @data to represent all the field + * @param fieldType type of this field + * @param sootField + */ + fun allocObjectField( + obj: PLObject, + fieldName: String, + fieldType: Type, + sootField: SootField? = null + ): PLPtrObjectField { + val ptrSig = getObjectFieldShortSignature(obj, fieldType, fieldName) + if (ptrIndexMap.containsKey(ptrSig)) { + return ptrIndexMap[ptrSig] as PLPtrObjectField + } + val ptr = PLPtrObjectField(obj, fieldName, fieldType, sootField, ptrSig) + ptrIndexMap[ptrSig] = ptr + return ptr + } + + /** + * get or create a PLPtrObjectField for @data + */ + fun allocObjectField( + obj: PLObject, + fieldName: String, + type: Type + ): PLPtrObjectField { + val elemType: Type = if (type is ArrayType) { + type.elementType + } else { + type + } + return allocObjectField(obj, fieldName, elemType, null) + } + + /** + * get or create a PLPtrStaticField + */ + fun allocStaticField(staticField: SootField): PLPtrStaticField { + val sig = staticField.shortSignature() + if (ptrIndexMap.containsKey(sig)) { + return ptrIndexMap[sig] as PLPtrStaticField + } + val ptr = PLPtrStaticField(staticField) + ptrIndexMap[sig] = ptr + return ptr + } + + companion object { + fun getObjectFieldSignature(obj: PLObject, fieldType: Type, fieldName: String): String { + return "pf{${obj.longSignature()}($fieldType)->$fieldName}" + } + + fun getObjectFieldShortSignature(obj: PLObject, fieldType: Type, fieldName: String): String { + return "pf{${obj.signature}(${fieldType.shortName()})->$fieldName}" + } + + fun typeWrapper(type: Type): Type { + return if (type is ArrayType && type.numDimensions > 1) { + ArrayType.v(type.baseType, 1) + } else type + } + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/pointer/readme.md b/src/main/kotlin/net/bytedance/security/app/pointer/readme.md new file mode 100644 index 0000000..c094801 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/pointer/readme.md @@ -0,0 +1,18 @@ +All possible statement types that taint propagates in Java: + +1. local to local + dst=src+b + src=dst +2. local to object field + o.dst=src +3. local to param + o.f(src,arg2,arg3) +4. local to this + dst=o.f() + +5. object field to local + dst=o.src +6. static field to local + dst=Object.src +7. local to static field + Object.dst=src diff --git a/src/main/kotlin/net/bytedance/security/app/preprocess/AnalyzePreProcessor.kt b/src/main/kotlin/net/bytedance/security/app/preprocess/AnalyzePreProcessor.kt new file mode 100644 index 0000000..34770f6 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/preprocess/AnalyzePreProcessor.kt @@ -0,0 +1,136 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.preprocess + +import kotlinx.coroutines.* +import net.bytedance.security.app.Log +import net.bytedance.security.app.PreAnalyzeContext +import net.bytedance.security.app.engineconfig.isLibraryClass +import net.bytedance.security.app.util.TaskQueue +import soot.Scene +import soot.SootClass +import soot.SootMethod + +/** +create the `Context` for rule processor. +it includes: +- method jimple SSA +- callback patching +- call graph building + */ +class AnalyzePreProcessor(private val parallelCount: Int, val ctx: PreAnalyzeContext) { + private val methodsVisitor = ArrayList>() + private val classesVisitor = ArrayList() + + /** + * You must add all the visitors before calling run + */ + fun addMethodVisitor(action: () -> MethodVisitor): AnalyzePreProcessor { + val thisTypeVisitors = ArrayList() + for (i in 0 until parallelCount) { + thisTypeVisitors.add(action()) + } + methodsVisitor.add(thisTypeVisitors) + return this + } + + fun addClassVisitor(action: () -> ClassVisitor): AnalyzePreProcessor { + classesVisitor.add(action()) + return this + } + + /** + * process of classes and methods + */ + suspend fun run() { + processClasses() + processMethods() + } + + + @Suppress("ControlFlowWithEmptyBody") + suspend fun processClasses() { + val classTasks = TaskQueue("classPreProcessor", parallelCount) { cls, _ -> + for (classVisitor in classesVisitor) { + for (m in cls.methods) { + // intentionally left blank + } + classVisitor.visitClass(cls) + } + } + val task = classTasks.runTask() + + try { + // Copy it to avoid class changes during traversal + val classList: List = ArrayList(Scene.v().classes) + for (cls in classList) { + if (isLibraryClass(cls.name)) { + continue + } + classTasks.addTask(cls) + } + } catch (e: Exception) { + e.printStackTrace() + throw e + } + classTasks.addTaskFinished() + task.join() + } + + + private suspend fun processMethods() { + Log.logInfo("processMethods........") + val methodTasks = + TaskQueue("methodPreProcessor", parallelCount) { method, index -> +// logDebug("process method ${method.signature}") + for (methodVisitor in methodsVisitor) { + methodVisitor[index].visitMethod(method) + } + } + + val task1 = methodTasks.runTask() + try { + val classList: List = ArrayList(Scene.v().classes) + for (cls in classList) { + if (isLibraryClass(cls.name)) { + continue + } + val methods = ArrayList(cls.methods) + //may conflict with method resolve + for (method in methods) { + methodTasks.addTask(method) + } + } + } catch (e: Exception) { +// e.printStackTrace() + throw e + } + methodTasks.addTaskFinished() + task1.join() + val scope = CoroutineScope(Dispatchers.Default) + val jobs = ArrayList() + for (v in methodsVisitor) { + val job = + scope.launch(CoroutineName("${this.javaClass.simpleName}-collect-${v.javaClass.simpleName}")) { + v[0].collect(v) + } + jobs.add(job) + } + jobs.joinAll() + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/preprocess/CallGraph.kt b/src/main/kotlin/net/bytedance/security/app/preprocess/CallGraph.kt new file mode 100644 index 0000000..3e5ae3a --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/preprocess/CallGraph.kt @@ -0,0 +1,463 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.preprocess + +import net.bytedance.security.app.Log +import net.bytedance.security.app.PLUtils +import net.bytedance.security.app.android.LifecycleConst +import net.bytedance.security.app.engineconfig.EngineConfig +import net.bytedance.security.app.engineconfig.isLibraryClass +import soot.SootClass +import soot.SootMethod +import java.util.* + +/** + * method call relations of all the app except library classes. + */ +class CallGraph { + // key is a caller method, value are all the direct callees of this method. + var directCallGraph: MutableMap> = HashMap() + + // key: a caller method value are all the direct callees of this method, + // consider all possible inheritance functions in terms of CHA relations + var heirCallGraph: MutableMap> = HashMap() + + // key:a callee method,value: all the direct callers of this method + var directReverseCallGraph: MutableMap> = HashMap() + + // key: a callee method value are all the direct callers of this method, + // consider all possible inheritance functions in terms of CHA relations + var heirReverseCallGraph: MutableMap> = HashMap() + + + private val traceCache: MutableMap = HashMap() + + fun clear() { + traceCache.clear() + } + + + private fun addToCallGraph(caller: SootMethod, callee: SootMethod, isDirect: Boolean) { + if (EngineConfig.libraryConfig.isLibraryMethod(caller.signature)) { + return + } + if (isDirect) { + val calleeSet = directCallGraph.computeIfAbsent(caller) { + HashSet() + } + calleeSet.add(callee) + val callerSet = directReverseCallGraph.computeIfAbsent(callee) { HashSet() } + callerSet.add(caller) + } + val calleeSet = heirCallGraph.computeIfAbsent(caller) { HashSet() } + calleeSet.add(callee) + val callerSet = heirReverseCallGraph.computeIfAbsent(callee) { HashSet() } + callerSet.add(caller) + } + + /** + * user method may have an empty body + */ + fun isUserCode(method: SootMethod): Boolean { + return !isLibraryClass(method.declaringClass.name) + } + + + fun addEdge(caller: SootMethod, callee: SootMethod, isDirect: Boolean) { + addToCallGraph(caller, callee, isDirect) + } + + + /* + which functions are called by entry at a given depth + */ + fun getAllCallees(entry: SootMethod, depth: Int): Set { + val s = HashSet() + query2Internal(entry, depth, s) + return s + } + + fun query2Internal(entry: SootMethod, depth: Int, s: HashSet) { + if (depth == 0) { + return + } + if (s.contains(entry)) { + return + } + s.add(entry) + + this.directCallGraph[entry]?.let { + for (m in it) { + query2Internal(m, depth - 1, s) + } + } + this.heirCallGraph[entry]?.let { + for (m in it) { + query2Internal(m, depth - 1, s) + } + } + } + + @Suppress("unused") + fun debugGetCalleesGraph(entry: SootMethod, depth: Int): Map> { + val s = HashMap>() + debugGetCalleesGraphInternal(entry, depth, s) + return s + } + + fun debugGetCalleesGraphInternal(entry: SootMethod, depth: Int, s: HashMap>) { + if (depth == 0) { + return + } + if (s.contains(entry.signature)) { + return + } + val cur = HashSet() + + this.directCallGraph[entry]?.let { + cur.addAll(it) + } + this.heirCallGraph[entry]?.let { + cur.addAll(it) + } + s[entry.signature] = cur.map { it.signature }.toSet() + for (m in cur) { + debugGetCalleesGraphInternal(m, depth - 1, s) + } + } + + fun queryTopEntry( + isPoly: Boolean, + callee: SootMethod, + depth: Int, + entrySet: MutableSet, + isPrintLog: Boolean = false + ) { + if (isPoly) { + queryTopEntryHeir(callee, depth, entrySet) + } else { + queryTopEntryDirect(callee, depth, entrySet, isPrintLog) + } + } + + + fun queryTopEntryNoCustomMain(isPoly: Boolean, callee: SootMethod, depth: Int, entrySet: MutableSet) { + if (isPoly) { + queryTopEntryHeirNoCustomMain(callee, depth, entrySet) + } else { + queryTopEntryDirectNoCustomMain(callee, depth, entrySet) + } + } + + /** + Find all call paths from topEntry to sink + */ + fun queryPath(topEntry: SootMethod, sink: SootMethod, depth: Int): MutableList { + val path: MutableList = ArrayList() + queryPathDirect(Stack(), topEntry, sink, path, depth) + return path + } + + private fun queryTopEntryDirect( + callee: SootMethod, + depth: Int, + entrySet: MutableSet, + isPrintLog: Boolean + ) { + if (depth == 0) { + entrySet.add(callee) + return + } + if (!directReverseCallGraph.containsKey(callee)) { + entrySet.add(callee) + return + } + if (isPrintLog) { + Log.logInfo("depth=$depth,nextCallee=${directReverseCallGraph[callee]}") + } + for (nextCallee in directReverseCallGraph[callee]!!) { + queryTopEntryDirect(nextCallee, depth - 1, entrySet, isPrintLog) + } + } + + private fun queryTopEntryDirectNoCustomMain( + callee: SootMethod, + depth2: Int, + entrySet: MutableSet, + ) { + var depth = depth2 + if (entrySet.contains(callee)) { + return + } + if (depth == 0) { + entrySet.add(callee) + return + } + if (!directReverseCallGraph.containsKey(callee)) { + entrySet.add(callee) + return + } + depth-- + for (nextCallee in directReverseCallGraph[callee]!!) { + if (nextCallee.signature.contains(PLUtils.CUSTOM_METHOD)) { + entrySet.add(callee) + continue + } + queryTopEntryDirectNoCustomMain(nextCallee, depth, entrySet) + } + } + + /** + Find all call paths from topEntry to sink + */ + private fun queryPathDirect( + stack: Stack, + topEntry: SootMethod, + sink: SootMethod, + path: MutableList, + maxDepth: Int + ) { + if (path.isNotEmpty()) { + stack.push(sink) + return + } + if (topEntry == sink) { + stack.push(topEntry) + path.addAll(stack) + return + } + if (stack.contains(topEntry)) { + stack.push(topEntry) + return + } + if (stack.size > maxDepth) { + stack.push(topEntry) + return + } + if (!directCallGraph.containsKey(topEntry)) { + stack.push(topEntry) + return + } + stack.push(topEntry) + for (nextTop in directCallGraph[topEntry]!!) { + queryPathDirect(stack, nextTop, sink, path, maxDepth) + stack.pop() + } + } + + /** + * Find the top function that calls callee indirectly, + * for example: f1->f2-> F3 ->f4->callee + * Unless the depth limit is reached or the method doesn't have any caller. + */ + private fun queryTopEntryHeir(callee: SootMethod, depth: Int, entrySet: MutableSet) { + if (depth == 0) { + entrySet.add(callee) + return + } + if (!heirReverseCallGraph.containsKey(callee)) { + entrySet.add(callee) + return + } + for (nextCallee in heirReverseCallGraph[callee]!!) { + queryTopEntryHeir(nextCallee, depth - 1, entrySet) + } + } + + private fun queryTopEntryHeirNoCustomMain( + callee: SootMethod, + depth: Int, + entrySet: MutableSet, + ) { + if (entrySet.contains(callee)) { + return + } + if (depth == 0) { + entrySet.add(callee) + return + } + if (!heirReverseCallGraph.containsKey(callee)) { + entrySet.add(callee) + return + } + for (nextCallee in heirReverseCallGraph[callee]!!) { + if (nextCallee.signature.contains(PLUtils.CUSTOM_METHOD)) { + entrySet.add(callee) + continue + } + queryTopEntryHeirNoCustomMain(nextCallee, depth - 1, entrySet) + } + } + + + /** + * find the first method which is the caller of sourceSig and sink,if there doesn't exist such caller,return null. + * @param polymorphism include polymorphism method or not + * @param source the source method + * @param sink the sink method + * @param depth max depth to search + * @return the caller to find + */ + fun traceAndCross(polymorphism: Boolean, source: SootMethod, sink: SootMethod, depth: Int): CrossResult? { + val cacheKey = source.signature + sink.signature + depth + synchronized(this) { + if (traceCache.containsKey(cacheKey)) { + return traceCache[cacheKey] + } + } + val ssc = SourceAndSinkCross(polymorphism, source, sink, depth, false, this) + val s = ssc.traceAndCross() + synchronized(this) { + traceCache[cacheKey] = s + } + return s + } + + + /** + top method is a method doesn't have a caller. + @return key is the class, value is the methods of this class that don't have a caller. + */ + fun getTopMethods(): Map> { + val r = HashMap>() + for ((caller, _) in this.heirCallGraph) { + if (this.heirReverseCallGraph.containsKey(caller)) { + continue + } + val c = caller.declaringClass + //skip android component + if (LifecycleConst.isComponentClass(c)) { + continue + } + //skip library + if (EngineConfig.libraryConfig.isLibraryClass(c.name)) { + continue + } + if (c.name.contains("$")) { + continue //skip internal class, because it's patched by MethodCallbackVisitor + } + var s = r[c] + if (s == null) { + s = HashSet() + r[c] = s + } + s.add(caller) + } + return r + } + + data class CrossResult(val entryMethod: SootMethod, val depth: Int) + + /** + * find a method where it's caller of src and caller of sink + algorithm: + if A represents the src, and B represents the sink + 1. S1=\[A\], S2=\[B\] + 2. if intersection of S1 and S2 is not empty, return the first method in the intersection + 3. add all the direct caller2 of A to S1, S1=[A,A11,A12,A13], + and add all the direct callers of B to S2, S2=[B,B11,B12,B13] + 4. go back to step 2 + */ + class SourceAndSinkCross( + poly: Boolean, + val src: SootMethod, + val sink: SootMethod, + var depth: Int, + private val isPrintLog: Boolean = false, + cg: CallGraph + ) { + val sourceSet = mutableSetOf(src) + val sinkSet = mutableSetOf(sink) + var srcNewSet = setOf(src) + var sinkNewSet = setOf(sink) + val queryMap: Map> + var result: CrossResult? = null + var resultMethod: SootMethod? = null + + init { + if (poly) { + queryMap = cg.heirReverseCallGraph + } else { + queryMap = cg.directReverseCallGraph + } + } + + fun traceAndCross(): CrossResult? { + crossInternal() + if (resultMethod != null) { + return CrossResult(resultMethod!!, depth) + } + return null + } + + /** + * don't use retainAll to avoid HashSet allocation + */ + private fun cross(s1: Set, s2: Set): SootMethod? { + for (a in s1) { + if (s2.contains(a)) { + return a + } + } + return null + } + + private fun crossInternal() { + while (true) { + if (depth == 0) { + return + } + resultMethod = cross(sourceSet, sinkSet) + if (resultMethod != null) { + return + } + val srcNewSet2 = HashSet() + val sinkNewSet2 = HashSet() + for (s in srcNewSet) { + queryMap[s]?.let { srcNewSet2.addAll(it) } + } + for (s in sinkNewSet) { + queryMap[s]?.let { sinkNewSet2.addAll(it) } + } + if (srcNewSet2.isEmpty() && sinkNewSet2.isEmpty()) { + return + } + sourceSet.addAll(srcNewSet2) + sinkSet.addAll(sinkNewSet2) + srcNewSet = srcNewSet2 + sinkNewSet = sinkNewSet2 + if (isPrintLog) { + Log.logInfo("depth=$depth,srcNewSet=$srcNewSet") + Log.logInfo("sinkNewSet=$sinkNewSet") + } + depth -= 1 + } + } + } + + companion object { + //for test only + fun toStringMap(m: Map>): Map> { + val dm: MutableMap> = HashMap() + m.map { entry -> + dm[entry.key.signature] = entry.value.map { it.signature }.toMutableSet() + } + return dm + } + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/preprocess/ClassCounter.kt b/src/main/kotlin/net/bytedance/security/app/preprocess/ClassCounter.kt new file mode 100644 index 0000000..3ba8b3f --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/preprocess/ClassCounter.kt @@ -0,0 +1,31 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.preprocess + +import net.bytedance.security.app.Log +import net.bytedance.security.app.PreAnalyzeContext +import soot.SootClass + +class ClassCounter(private val ctx: PreAnalyzeContext) : ClassVisitor { + override fun visitClass(cls: SootClass) { + val n = ctx.addClassCounter() + if (n % 100 == 0) { + Log.logInfo("processed $n classes") + } + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/preprocess/ClassVisitor.kt b/src/main/kotlin/net/bytedance/security/app/preprocess/ClassVisitor.kt new file mode 100644 index 0000000..296eea9 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/preprocess/ClassVisitor.kt @@ -0,0 +1,32 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.preprocess + +import soot.SootClass + +/** + * visit a sootClass + */ +interface ClassVisitor { + /** + * Access a specific SootClass + * + * @param cls + */ + fun visitClass(cls: SootClass) // void visitMethodSSA(SootMethod method); +} diff --git a/src/main/kotlin/net/bytedance/security/app/preprocess/MethodCallbackVisitor.kt b/src/main/kotlin/net/bytedance/security/app/preprocess/MethodCallbackVisitor.kt new file mode 100644 index 0000000..0db0251 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/preprocess/MethodCallbackVisitor.kt @@ -0,0 +1,62 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.preprocess + +import net.bytedance.security.app.Log.logErr +import soot.SootMethod + +/** +call back of + */ +class MethodCallbackVisitor(private val callbackEnhance: Boolean) : MethodVisitor { + override fun visitMethod(method: SootMethod) { + if (!method.hasActiveBody()) { + return + } + try { + doVisit(method) + } catch (e: Exception) { + logErr("this exception only related to preprocess of method $method") + e.printStackTrace() + } + } + + /** + * 1. patch for + * 2. callback from config file + * 3. patchFindviewByIdForWebview + * 4.Reflection + */ + private fun doVisit(method: SootMethod) { + Patch.patchCLInit(method) + val before = method.activeBody.units.size + MethodPatch.processCallback( + method, + callbackEnhance + ) + val after = method.activeBody.units.size + if (after > before) { + //if there are changes, generate jimple ssa again + MethodSSAVisitor.jimpleSSAPreprocess(method) + } + } + + override fun collect(visitors: List) { + //do nothing + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/preprocess/MethodCounter.kt b/src/main/kotlin/net/bytedance/security/app/preprocess/MethodCounter.kt new file mode 100644 index 0000000..d63bf09 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/preprocess/MethodCounter.kt @@ -0,0 +1,39 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.preprocess + +import net.bytedance.security.app.Log +import net.bytedance.security.app.PreAnalyzeContext +import soot.SootMethod + +class MethodCounter(val ctx: PreAnalyzeContext) : MethodVisitor { + override fun visitMethod(method: SootMethod) { +// PLLog.logInfo("visit visitMethod: ${method.signature}") + if (!method.isConcrete) { + return + } + val n = ctx.addMethodCounter() + if (n % 1000 == 0) { + Log.logInfo("processed $n methods") + } + } + + override fun collect(visitors: List) { + //do nothing + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/preprocess/MethodFieldConstCacheVisitor.kt b/src/main/kotlin/net/bytedance/security/app/preprocess/MethodFieldConstCacheVisitor.kt new file mode 100644 index 0000000..992ee3c --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/preprocess/MethodFieldConstCacheVisitor.kt @@ -0,0 +1,332 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.preprocess + +import net.bytedance.security.app.Log +import net.bytedance.security.app.PLUtils +import net.bytedance.security.app.PreAnalyzeContext +import net.bytedance.security.app.engineconfig.EngineConfig +import net.bytedance.security.app.engineconfig.isLibraryClass +import soot.* +import soot.jimple.* +import soot.jimple.internal.JAssignStmt +import soot.jimple.internal.JNewExpr +import soot.jimple.internal.JSpecialInvokeExpr +import soot.jimple.internal.JStaticInvokeExpr +import java.util.* + +/** + * Collect: + * 1. all function access, + * 2. constants from the rule file + * 3. field access from rule file + * 4. new instance from rule file + * 5. generate call graph for the whole app + */ +class MethodFieldConstCacheVisitor( + val ctx: PreAnalyzeContext, + val cache: MethodStmtFieldCache, + private val constStrPatternFilter: Set, + private val filedFilter: Set, + private val newInstanceFilter: Set +) : + MethodVisitor { + override fun visitMethod(method: SootMethod) { + if (!method.isConcrete || !method.hasActiveBody()) { + return + } + visitEachStmt(method, method.activeBody.units) + } + + /** + * Merge the results into ctx and generate the call graph + */ + override fun collect(visitors: List) { + collectMethodInvoke(visitors) + collectOthers(visitors) + } + + /** + * create call graph + */ + private fun collectMethodInvoke(visitors: List) { + for (v in visitors) { + val vv = v as MethodFieldConstCacheVisitor + for ((callee, value) in vv.cache.methodDirectRefs) { + val globalMethodCache = ctx.methodDirectRefs + for (caller in value) { + ctx.callGraph.addEdge(caller.method, callee, true) + } + if (ctx.methodDirectRefs.containsKey(callee)) { + globalMethodCache[callee]!!.addAll(value) + } else { + globalMethodCache[callee] = value + } + } + for ((callee, value) in vv.cache.methodHeirRefs) { + for (caller in value) { + ctx.callGraph.addEdge(caller.method, callee, false) + } + } + } + } + + + private fun collectOthers(visitors: List) { + for (v in visitors) { + val vv = v as MethodFieldConstCacheVisitor + for ((key, value) in vv.cache.loadFieldRefs) { + if (ctx.loadFieldRefs.containsKey(key)) { + ctx.loadFieldRefs[key]!!.addAll(value) + } else { + ctx.loadFieldRefs[key] = value + } + } + for ((key, value) in vv.cache.storeFieldRefs) { + if (ctx.storeFieldRefs.containsKey(key)) { + ctx.storeFieldRefs[key]!!.addAll(value) + } else { + ctx.storeFieldRefs[key] = value + } + } + for ((key, value) in vv.cache.newInstanceRefs) { + val clz = key.sootClass + if (ctx.newInstanceRefs.contains(clz)) { + ctx.newInstanceRefs[clz]!!.addAll(value) + } else { + ctx.newInstanceRefs[clz] = value + } + } + for ((key, value) in vv.cache.constStringPatternMap) { + if (ctx.constStringPatternMap.containsKey(key)) { + ctx.constStringPatternMap[key]!!.addAll(value) + } else { + ctx.constStringPatternMap[key] = value + } + } + } + } + + private fun visitEachStmt(sootMethod: SootMethod, chain: UnitPatchingChain) { + for (unit in chain) { + val stmt = unit as Stmt + val constStrings = getConstStringFromStmt(stmt) + for (constString in constStrings) { + for (pattern in constStrPatternFilter) { + if (PLUtils.isStrMatch(pattern, constString)) { + cache.addPattern(pattern, sootMethod, stmt) + } + } + } + if (stmt.containsInvokeExpr()) { + addMethodInvoke(stmt.invokeExpr, sootMethod, stmt) + } else if (stmt is JAssignStmt) { + val leftExpr = stmt.leftOp + val rightExpr = stmt.rightOp + if (rightExpr is JNewExpr) { + val typ = rightExpr.type as RefType + if (newInstanceFilter.contains(typ.className)) { + cache.addNewInstanceCache(typ, sootMethod, stmt) + } + } else { + val (field, isStore) = if (rightExpr is StaticFieldRef) { + val field = rightExpr.fieldRef.resolve() + Pair(field, false) + } else if (leftExpr is StaticFieldRef) { + val field = leftExpr.fieldRef.resolve() + Pair(field, true) + } else if (rightExpr is InstanceFieldRef) { + val field = rightExpr.fieldRef.resolve() + Pair(field, false) + } else if (leftExpr is InstanceFieldRef) { + val field = leftExpr.fieldRef.resolve() + Pair(field, true) + } else { + continue + } + if (field == null) { + Log.logWarn("cannot found field ${stmt}") + continue + } + + if (!filedFilter.contains(field.signature)) { + continue + } + if (isStore) { + cache.addStoreFieldCache(field, sootMethod, stmt) + } else { + cache.addLoadFieldCache(field, sootMethod, stmt) + } + + } + } + } + } + + private fun addMethodInvoke(invokeExpr: InvokeExpr, sootMethod: SootMethod, stmt: Stmt) { + val ref = invokeExpr.methodRef + val m = try { + ref.resolve() + } catch (ex: Exception) { + throw RuntimeException("resolve method exception,method:${ref.signature}") + } + if (m == null) { + Log.logInfo("resolve method error,method:${ref.signature}") + return + } + cache.addMethodDirectCache(m, sootMethod, stmt) + if (invokeExpr is JSpecialInvokeExpr || invokeExpr is JStaticInvokeExpr) { + return + } + if (!canMethodHasSubMethods(m)) { + return + } + if (isLibraryClass(m.declaringClass.name)) { + return + } + getAllHeirMethods(ref)?.let { methods -> + methods.forEach { method -> + cache.addMethodHeirCache(method, sootMethod, stmt) + } + } + } + + companion object { + + + fun getAllHeirMethods(method: SootMethodRef): MutableSet? { + if (isLibraryClass(method.declaringClass.name)) + return null + val resolvedMethod = method.resolve() + val possibleSet: MutableSet = HashSet() + getAllHeirClassesWithSubMethodSig(method.declaringClass, possibleSet, resolvedMethod.subSignature) + // PLLog.logErr(methodSig+" possible classes "+possibleSet.toString()); + if (possibleSet.isNotEmpty()) { + val subHeirCalleeSet: MutableSet = HashSet() + for (sc in possibleSet) { + val m = sc.getMethodUnsafe(method.subSignature) + if (m != null) { + subHeirCalleeSet.add(m) + } else { + Log.logInfo("getAllHeirMethods method error, method:$method, class:$sc") + } + } + subHeirCalleeSet.remove(resolvedMethod) + return subHeirCalleeSet + } + return null + } + + private fun getAllHeirClassesWithSubMethodSig( + sc: SootClass, + possibleClass: MutableSet, + subMethodSig: String + ) { + val stack = Stack() + + if (!sc.declaresMethod(subMethodSig)) { + getAllSuperClasses(sc, stack, possibleClass, subMethodSig) + } + getAllSubClasses(sc, stack, possibleClass, subMethodSig) + } + + private fun getAllSubClasses( + sc: SootClass, + stack: Stack, + superClasses: MutableSet, + subMethodSig: String + ) { + stack.push(sc) + if (sc.declaresMethod(subMethodSig)) { + val m = sc.getMethodUnsafe(subMethodSig) + if (m != null && m.isConcrete) { + superClasses.add(sc) + } + } + if (stack.size > 8) { + return + } + if (sc.isInterface) { + val subClassSet = Scene.v().orMakeFastHierarchy.getAllImplementersOfInterface(sc) + if (subClassSet != null) { + for (sootClass in subClassSet) { + if (EngineConfig.libraryConfig.isLibraryClass(sootClass.name)) { + continue + } + getAllSubClasses(sootClass, stack, superClasses, subMethodSig) + stack.pop() + } + } + } else { + val subClassSet = Scene.v().orMakeFastHierarchy.getSubclassesOf(sc) + if (subClassSet != null) { + for (sootClass in subClassSet) { + getAllSubClasses(sootClass, stack, superClasses, subMethodSig) + stack.pop() + } + } + } + } + + private fun getAllSuperClasses( + sc: SootClass, + stack: Stack, + superClasses: MutableSet, + subMethodSig: String + ) { + stack.push(sc) + if (EngineConfig.libraryConfig.isLibraryClass(sc.name)) { + return + } + if (superClasses.size > 0) { + return + } + if (sc.declaresMethod(subMethodSig)) { + if (sc.getMethodUnsafe(subMethodSig).isConcrete) { + superClasses.add(sc) + } + return + } + if (stack.size > 8) { + return + } + if (sc.hasSuperclass()) { + val sootClass = sc.superclass + getAllSuperClasses(sootClass, stack, superClasses, subMethodSig) + stack.pop() + } + } + + fun getConstStringFromStmt(stmt: Stmt): List { + val lists = ArrayList() + for (valueBox in stmt.useAndDefBoxes) { + val value = valueBox.value + if (value is StringConstant) { + val constStr = value.value + lists.add(constStr) + } + } + return lists + } + + fun canMethodHasSubMethods(method: SootMethod): Boolean { + return !(method.isConstructor || method.isStaticInitializer || method.isFinal || method.isNative || method.isPrivate || method.isFinal || method.isStatic) + } + } + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/preprocess/MethodPatch.kt b/src/main/kotlin/net/bytedance/security/app/preprocess/MethodPatch.kt new file mode 100644 index 0000000..06b060f --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/preprocess/MethodPatch.kt @@ -0,0 +1,186 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.preprocess + +import net.bytedance.security.app.Log +import net.bytedance.security.app.PLUtils +import net.bytedance.security.app.engineconfig.EngineConfig + +import soot.* +import soot.Unit +import soot.jimple.* + +/** + * 1. do callback patch from config file + * 2. do reflection patch + * 3. do patch for patchFindviewByIdForWebview + */ +object MethodPatch { + private var callBackEnhance = false + + + private val floatConstant = FloatConstant.v(3f) + private val doubleConstant = DoubleConstant.v(2.0) + private val intConstant = IntConstant.v(1) + private val nullConstant = NullConstant.v() + private fun createNewInvokeUnit(sm: SootMethod, base: Value): Unit? { + val args: MutableList = ArrayList() + for (type in sm.parameterTypes) { + if (type is PrimType) { + if (type is FloatType) { + args.add(floatConstant) + } else if (type is DoubleType) { + args.add(doubleConstant) + } else { + args.add(intConstant) + } + } else { + args.add(nullConstant) + } + } + try { + return if (sm.isStatic) { + Jimple.v().newInvokeStmt(Jimple.v().newStaticInvokeExpr(sm.makeRef(), args)) + } else { + Jimple.v().newInvokeStmt(Jimple.v().newVirtualInvokeExpr(base as Local, sm.makeRef(), args)) + } + } catch (e: Exception) { + e.printStackTrace() + } + return null + } + + private fun patchForStmt(stmt: Stmt, nextStmt: Stmt?, method: SootMethod, patchUnits: MutableList) { + calcCallbackPatchUnits(stmt, patchUnits) + Reflection.tryInject(stmt, method, patchUnits) + Patch.patchFindviewByIdForWebview(stmt, nextStmt, method, patchUnits) + } + + /** + * do patch for constructor function + * 1. patch from Callback config file, for example if method F create a new instance of android.os.Handler, + * call the handleMessage + * 2. if callBackEnhance is true, call all the internal class's function member. + */ + private fun calcCallbackPatchUnits(stmt: Stmt, patchUnits: MutableList) { + val invokeExpr = stmt.invokeExpr as? InstanceInvokeExpr ?: return + if (invokeExpr.methodRef.isConstructor) { + val base = invokeExpr.base + val declMethod = + try { + Patch.resolveMethodException(invokeExpr) + } catch (ex: Exception) { + Log.logInfo("calcCallbackPatchUnits: stmt=${stmt}") + Log.logInfo("calcCallbackPatchUnits ex= ${ex.stackTraceToString()}") + throw ex + } + val declClass = declMethod.declaringClass + val baseCls = Scene.v().getSootClassUnsafe(base.type.toString(), false) + if (declClass == baseCls && EngineConfig.callbackConfig.getCallBackConfig().containsKey(baseCls)) { + for (subMethodSig in EngineConfig.callbackConfig.getCallBackConfig()[baseCls]!!) { + val sm = PLUtils.dispatchCall(baseCls, subMethodSig) + if (sm == null || !sm.isConcrete) { + continue + } + if (sm.declaringClass != declClass) { + continue + } + val newUnit = createNewInvokeUnit(sm, base) + if (newUnit != null) { + patchUnits.add(newUnit) + } + } + } else if (callBackEnhance && baseCls.isInnerClass) { + val classInterfaces = baseCls.interfaces + val subMethodSet: MutableSet = HashSet() + for (classInterface in classInterfaces) { + if (EngineConfig.callbackConfig.enhanceIgnore.contains(classInterface.name)) { + continue + } + for (method in classInterface.methods) { + subMethodSet.add(method.subSignature) + } + } + if (baseCls.hasSuperclass()) { + val superClass = baseCls.superclass + if (superClass.name != "java.lang.Object") { + for (superMethod in superClass.methods) { + subMethodSet.add(superMethod.subSignature) + } + } + } + if (subMethodSet.isEmpty()) { + return + } + for (sm in baseCls.methods) { + if (sm.isConstructor || sm.isStaticInitializer) { + continue + } + if (!subMethodSet.contains(sm.subSignature)) { + continue + } + val newUnit = createNewInvokeUnit(sm, base) + if (newUnit != null) { + patchUnits.add(newUnit) + } + } + } + } + } + + private fun injectAll( + method: SootMethod, + stmt: Stmt, + nextStmt: Stmt?, + methodUnits: UnitPatchingChain, + iterator: MutableListIterator, + ) { + if (stmt.containsInvokeExpr()) { + val patchUnits: MutableList = ArrayList() + patchForStmt(stmt, nextStmt, method, patchUnits) + if (patchUnits.isNotEmpty()) { + methodUnits.insertAfter(patchUnits, stmt) + for (patchUnit in patchUnits) { + iterator.add(patchUnit) + iterator.previous() + } + iterator.next() + } + } + } + + fun processCallback( + method: SootMethod, + isCallBackEnhance: Boolean + ) { + callBackEnhance = isCallBackEnhance + val tmpChain: MutableList = ArrayList(method.activeBody.units) + val iterator = tmpChain.listIterator() + while (iterator.hasNext()) { + val stmt = iterator.next() as Stmt + val nextStmt = if (iterator.hasNext()) { + val stmt2 = iterator.next() as Stmt + iterator.previous() + stmt2 + } else { + null + } + injectAll(method, stmt, nextStmt, method.activeBody.units, iterator) + } + } +} diff --git a/src/main/kotlin/net/bytedance/security/app/preprocess/MethodSSAVisitor.kt b/src/main/kotlin/net/bytedance/security/app/preprocess/MethodSSAVisitor.kt new file mode 100644 index 0000000..d92f4d1 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/preprocess/MethodSSAVisitor.kt @@ -0,0 +1,65 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.preprocess + +import net.bytedance.security.app.Log.logErr +import soot.Body +import soot.PackManager +import soot.SootMethod +import soot.shimple.Shimple +import soot.shimple.ShimpleBody + +/** + * + */ +class MethodSSAVisitor : MethodVisitor { + override fun visitMethod(method: SootMethod) { + if (!method.isConcrete) { + return + } + try { + jimpleSSAPreprocess(method) + } catch (e: Exception) { + logErr("JimpPreprocess error: $e,for method:${method.signature}") + } + } + + override fun collect(visitors: List) { + // + } + + + companion object { + fun jimpleSSAPreprocess(sootMethod: SootMethod): Body { + val sBody: ShimpleBody + val body = sootMethod.retrieveActiveBody() + if (body is ShimpleBody) { + sBody = body + if (!sBody.isSSA) sBody.rebuild() + } else { + sBody = Shimple.v().newBody(body) + } + sootMethod.activeBody = sBody + PackManager.v().getPack("stp").apply(sBody) + PackManager.v().getPack("sop").apply(sBody) + sootMethod.activeBody = sBody.toJimpleBody() + return sootMethod.activeBody + } + + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/preprocess/MethodStmtFieldCache.kt b/src/main/kotlin/net/bytedance/security/app/preprocess/MethodStmtFieldCache.kt new file mode 100644 index 0000000..c3128bd --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/preprocess/MethodStmtFieldCache.kt @@ -0,0 +1,102 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.preprocess + +import soot.RefType +import soot.SootField +import soot.SootMethod +import soot.jimple.Stmt + +class CallSite(val method: SootMethod, val stmt: Stmt) { + /** + * r1="aaa" + * r1=o.f("a1","a2,"a3") + * Extract a constant string from stmt above + */ + fun constString(): List { + return MethodFieldConstCacheVisitor.getConstStringFromStmt(stmt) + } + + override fun toString(): String { + return "CallSite(method=$method, stmt=$stmt)" + } +} + +class MethodStmtFieldCache { + + /** + * key is the callee function,value is the caller and it's callsite + * Direct is a function called directly, regardless of the CHA relationship + */ + val methodDirectRefs: MutableMap> = HashMap() + val methodHeirRefs: MutableMap> = HashMap() + + /** + * key is the field to load,value are callsites + * a=b.c; + */ + val loadFieldRefs: MutableMap> = HashMap() + + /** + * key is the field to store,value are callsites + * b.c=a; + */ + val storeFieldRefs: MutableMap> = HashMap() + + /** + + key if const string pattern from rule file,value is the callsite + */ + var constStringPatternMap: MutableMap> = HashMap() + + /** + * key is a class,value is callsite of new instance + */ + val newInstanceRefs: MutableMap> = HashMap() + fun addMethodDirectCache(key: SootMethod, method: SootMethod, stmt: Stmt) { + val cache = methodDirectRefs.computeIfAbsent(key) { HashSet() } + cache.add(CallSite(method, stmt)) + } + + fun addMethodHeirCache(key: SootMethod, method: SootMethod, stmt: Stmt) { + val cache = methodHeirRefs.computeIfAbsent(key) { HashSet() } + cache.add(CallSite(method, stmt)) + } + + fun addLoadFieldCache(key: SootField, method: SootMethod, stmt: Stmt) { + val cache = loadFieldRefs.computeIfAbsent(key) { HashSet() } + cache.add(CallSite(method, stmt)) + } + + fun addStoreFieldCache(key: SootField, method: SootMethod, stmt: Stmt) { + val cache = storeFieldRefs.computeIfAbsent(key) { HashSet() } + cache.add(CallSite(method, stmt)) + } + + fun addNewInstanceCache(key: RefType, method: SootMethod, stmt: Stmt) { + val cache = newInstanceRefs.computeIfAbsent(key) { HashSet() } + cache.add(CallSite(method, stmt)) + } + + fun addPattern(pattern: String, method: SootMethod, stmt: Stmt) { + val cache = constStringPatternMap.computeIfAbsent(pattern) { HashSet() } + cache.add(CallSite(method, stmt)) + } + + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/preprocess/MethodVisitor.kt b/src/main/kotlin/net/bytedance/security/app/preprocess/MethodVisitor.kt new file mode 100644 index 0000000..8028599 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/preprocess/MethodVisitor.kt @@ -0,0 +1,35 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.preprocess + +import soot.SootMethod + + +/** + * process of a SootMethod + */ +interface MethodVisitor { + + fun visitMethod(method: SootMethod) + + /** + * after processing all the methods, + * call collect to classify + */ + fun collect(visitors: List) +} diff --git a/src/main/kotlin/net/bytedance/security/app/preprocess/Patch.kt b/src/main/kotlin/net/bytedance/security/app/preprocess/Patch.kt new file mode 100644 index 0000000..592c500 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/preprocess/Patch.kt @@ -0,0 +1,111 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.preprocess + +import net.bytedance.security.app.Log.logErr +import soot.Scene +import soot.SootMethod +import soot.Unit +import soot.jimple.InvokeExpr +import soot.jimple.Jimple +import soot.jimple.Stmt +import soot.jimple.internal.JAssignStmt +import soot.jimple.internal.JCastExpr +import soot.jimple.internal.JimpleLocal + +/* +do patch in preprocess stage + */ +object Patch { + /** + * If a Class has , it must be added to each function + */ + fun patchCLInit(m: SootMethod) { + if (!m.isConstructor) { + return + } + for (method in m.declaringClass.methods) { + if (!method.isStaticInitializer) { + continue + } + val stmt: Stmt = Jimple.v().newInvokeStmt( + // invoke + Jimple.v().newStaticInvokeExpr(method.makeRef()) + ) + m.activeBody.units.insertAfter(stmt, m.activeBody.units.last) + } + } + + /** + for rule like "NewInstance": ["android.webkit.WebView"] + $r2 = virtualinvoke r0.(2131231042); + r3 = (android.webkit.WebView) $r2; + insert a statement: r3=new android.webkit.Webview; + */ + fun patchFindviewByIdForWebview( + stmt: Stmt, + nextStmt: Stmt?, + @Suppress("UNUSED_PARAMETER") method: SootMethod, + patchUnits: MutableList + ) { + val locals = ArrayList() + val stmts = ArrayList() + if (nextStmt == null) { + return + } + if (!stmt.containsInvokeExpr()) { + return + } + val invokeExpr = stmt.invokeExpr + if (resolveMethodException(invokeExpr).signature.indexOf("android.view.View findViewById(int)>") < 0) { + return + } + + val stmt2 = nextStmt as? JAssignStmt ?: return + if (stmt2.rightOp !is JCastExpr) { + return + } + if (stmt2.leftOp !is JimpleLocal) { + return + } + val leftExpr = stmt2.leftOp as JimpleLocal + val right = stmt2.rightOp as JCastExpr + if (right.castType.toString() != "android.webkit.WebView") { + return + } + locals.add(leftExpr) + stmts.add(stmt2) + for (i in locals.indices) { + val j = locals[i] + val sootClass = Scene.v().getSootClassUnsafe(j.type.toString(), false) + val newExpr = Jimple.v().newNewExpr(sootClass.type) + val assignStmt = Jimple.v().newAssignStmt(j, newExpr) + patchUnits.add(assignStmt) + } + } + + @Synchronized + fun resolveMethodException(expr: InvokeExpr): SootMethod { + try { + return expr.method + } catch (e: Exception) { + logErr("resolve $expr failed") + throw e + } + } +} diff --git a/src/main/kotlin/net/bytedance/security/app/preprocess/Reflection.kt b/src/main/kotlin/net/bytedance/security/app/preprocess/Reflection.kt new file mode 100644 index 0000000..a0b3e4f --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/preprocess/Reflection.kt @@ -0,0 +1,635 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.preprocess + +import net.bytedance.security.app.Log.logErr +import net.bytedance.security.app.preprocess.Patch.resolveMethodException +import soot.* +import soot.Unit +import soot.jimple.* +import soot.jimple.internal.* + +/** + * patches for reflection +public void reflectionCall() throws ClassNotFoundException, NoSuchMethodException, InvocationTargetException, InstantiationException, IllegalAccessException { +Class clz = Class.forName("net.bytedance.security.app.preprocess.testdata.Apple"); +Method setPriceMethod = clz.getMethod("setPrice", int.class); +Constructor appleConstructor = clz.getConstructor(); +Object appleObj = appleConstructor.newInstance(); +setPriceMethod.invoke(appleObj, 14); +Method getPriceMethod = clz.getMethod("getPrice"); +System.out.println("Apple Price:" + getPriceMethod.invoke(appleObj)); +} +find the concrete class and call the method + */ +object Reflection { + private const val REFLECT_FOR_NAME = "" + private const val REFLECT_GET_CLASS = "" + private const val REFLECT_GET_METHOD_SIG = + "" + private const val REFLECT_GET_DECLARE_METHOD_SIG = + "" + private const val INVOKE_METHOD_SIG = + "" + + fun tryInject(stmt: Stmt, caller: SootMethod, patchUnits: MutableList) { + val invokeExpr = stmt.invokeExpr + when (invokeExpr.methodRef.signature) { + REFLECT_FOR_NAME -> { + injectReflectForName(stmt, caller, patchUnits) + } + REFLECT_GET_CLASS -> { + injectReflectGetClass(stmt, caller, patchUnits) + } + INVOKE_METHOD_SIG -> { + injectXXClass(stmt, caller, patchUnits) + } + } + } + + + private fun injectReflectForName(stmt: Stmt, caller: SootMethod, patchUnits: MutableList) { + val arrayMap = arrayAnalyze(caller) + val forNameRetMap = insertInstantStmt(stmt, caller, patchUnits) + if (forNameRetMap.isEmpty()) { + return + } + val getMethodRetMap = calcGetMethod(caller, arrayMap, forNameRetMap) + if (getMethodRetMap.isEmpty()) { + return + } + injectInvokeStmt(caller, arrayMap, getMethodRetMap, patchUnits) + } + + private fun injectReflectGetClass(stmt: Stmt, caller: SootMethod, patchUnits: MutableList) { + val arrayMap = arrayAnalyze(caller) + val forNameRetMap = insertGetClassInstantStmt(stmt, caller, patchUnits) + if (forNameRetMap.isEmpty()) { + return + } + val getMethodRetMap = calcGetMethod(caller, arrayMap, forNameRetMap) + if (getMethodRetMap.isEmpty()) { + return + } + injectInvokeStmt(caller, arrayMap, getMethodRetMap, patchUnits) + } + + //xx.class() + private fun injectXXClass( + @Suppress("UNUSED_PARAMETER") stmt: Stmt, + caller: SootMethod, + patchUnits: MutableList + ) { + if (!isXxClassReflection(caller.toString())) { + return + } + val arrayMap = arrayAnalyze(caller) + val forNameRetMap: Map> = insertxxClassInstantStmt(stmt, caller) + if (forNameRetMap.isEmpty()) { + return + } + val getMethodRetMap = calcGetMethod(caller, arrayMap, forNameRetMap) + if (getMethodRetMap.isEmpty()) { + return + } + injectInvokeStmt(caller, arrayMap, getMethodRetMap, patchUnits) + } + + + private fun injectInvokeStmt( + entryMethod: SootMethod, + arrayMap: Map>, + getMethodRetMap: Map>, + patchUnits: MutableList + ) { + val stmtInvokeMap: MutableMap = HashMap() + for (unit in entryMethod.activeBody.units) { + val stmt = unit as Stmt + if (!stmt.containsInvokeExpr()) { + continue + } + val invokeExpr = stmt.invokeExpr as? InstanceInvokeExpr ?: continue + val baseVal = invokeExpr.base as? JimpleLocal ?: continue + if (!getMethodRetMap.containsKey(baseVal)) { + continue + } + val reflectInstanceInvoke = getMethodRetMap[baseVal]!! + var retLocal: JimpleLocal? = null + if (stmt is JAssignStmt) { + val left = stmt.leftOp + if (left is JimpleLocal) { + retLocal = left + } + } + val invokeSig = invokeExpr.methodRef.signature + if (invokeSig == INVOKE_METHOD_SIG) { +// Value classVal = instanceInvokeExpr.getArg(0); + val argArrVal = invokeExpr.getArg(1) as? JimpleLocal ?: continue + var methodArgs: List? = ArrayList() + if (arrayMap.containsKey(argArrVal)) { + methodArgs = arrayMap[argArrVal] + } + val reflectDispatch = if (reflectInstanceInvoke.second.isStatic) { + Jimple.v().newStaticInvokeExpr(reflectInstanceInvoke.second.makeRef(), methodArgs) + } else { + val sc = reflectInstanceInvoke.second.makeRef().declaringClass + if (sc.isInterface) { + Jimple.v().newInterfaceInvokeExpr( + reflectInstanceInvoke.first, + reflectInstanceInvoke.second.makeRef(), + methodArgs + ) + } else { + Jimple.v().newVirtualInvokeExpr( + reflectInstanceInvoke.first, + reflectInstanceInvoke.second.makeRef(), methodArgs + ) + } + } + if (retLocal != null) { + val assign = Jimple.v().newAssignStmt(retLocal, reflectDispatch) + stmtInvokeMap[assign] = stmt + } else { + val invokeStmt = Jimple.v().newInvokeStmt(reflectDispatch) + stmtInvokeMap[invokeStmt] = stmt + } + } + } + patchUnits.addAll(stmtInvokeMap.keys.toList()) + } + + private fun calcGetMethod( + entryMethod: SootMethod, + arrayMap: Map>, + forNameRetMap: Map> + ): Map> { + val getMethodRetMap: MutableMap> = HashMap() + for (unit in entryMethod.activeBody.units) { + val stmt = unit as Stmt + if (!stmt.containsInvokeExpr()) { + continue + } + val invokeExpr = stmt.invokeExpr as? InstanceInvokeExpr ?: continue + val baseVal = invokeExpr.base as? JimpleLocal ?: continue + var retLocal: JimpleLocal? = null + if (stmt is JAssignStmt) { + val left = stmt.leftOp + if (left is JimpleLocal) { + retLocal = left + } + } + val invokeSig = invokeExpr.methodRef.signature + if (invokeSig == REFLECT_GET_METHOD_SIG || invokeSig == REFLECT_GET_DECLARE_METHOD_SIG) { + if (!forNameRetMap.containsKey(baseVal)) { + continue + } + val nameVal = invokeExpr.getArg(0) + val argArrVal = invokeExpr.getArg(1) + if (nameVal !is StringConstant) { + continue + } + if (argArrVal !is JimpleLocal) { + continue + } + val methodName = nameVal.value + var methodArgs: List? = null + if (arrayMap.containsKey(argArrVal)) { + methodArgs = arrayMap[argArrVal] + } + val reflectMethod = getMethod(forNameRetMap[baseVal]!!, methodName, methodArgs) + if (reflectMethod != null) { + getMethodRetMap[retLocal] = Pair(forNameRetMap[baseVal]!!.first, reflectMethod) + } + } + } + return getMethodRetMap + } + + private fun getMethod( + instanceClass: Pair, + methodName: String, + methodArgs: List? + ): SootMethod? { + for (classMethod in instanceClass.second.methods) { + if (classMethod.name == methodName) { + var isAllParamEqual = true + val sz = methodArgs?.size ?: 0 + if (sz != classMethod.parameterCount) { + isAllParamEqual = false + } + if (isAllParamEqual) { + return classMethod + } + } + } + return null + } + + /* + + r1 = newarray (java.lang.Class)[5]; + + r1[0] = class "Ljava/lang/String;"; + + r1[1] = class "Ljava/lang/String;"; + + r1[2] = class "Ljava/lang/String;"; + + r1[3] = class "Landroid/app/PendingIntent;"; + + r1[4] = class "Landroid/app/PendingIntent;"; +m + $r3 = virtualinvoke $r2.("sendTextMessage", r1); + + $r4 = newarray (java.lang.Object)[5]; + + $r4[0] = "13966668888"; + + $r4[1] = "110"; + + $r4[2] = "hello world!"; + + $r4[3] = null; + + $r4[4] = null; + */ + private fun arrayAnalyze(entryMethod: SootMethod): Map> { + val arrayMap: MutableMap> = HashMap() + for (unit in entryMethod.activeBody.units) { + val stmt = unit as Stmt + if (stmt is JAssignStmt) { + val left = stmt.leftOp + val right = stmt.rightOp + if (left is JArrayRef) { + val baseLocal = left.base as JimpleLocal + if (arrayMap.containsKey(baseLocal)) { + arrayMap[baseLocal]!!.add(right) + } + } else if (left is JimpleLocal) { + val leftLocal = stmt.leftOp as JimpleLocal + if (right is JNewArrayExpr) { + arrayMap[leftLocal] = ArrayList() + } + } + } + } + return arrayMap + } + + private fun insertInstantStmt( + forNameStmt: Stmt, + entryMethod: SootMethod, + patchUnits: MutableList + ): Map> { + var localCnt = entryMethod.activeBody.localCount + val reflectMap: MutableMap> = HashMap() + + if (forNameStmt !is JAssignStmt) { + return reflectMap + } + val forNameInvoke = forNameStmt.invokeExpr + val classNameArg = forNameInvoke.getArg(0) as? StringConstant ?: return reflectMap + // + val reflectClassName = classNameArg.value + val reflectClass: SootClass? = try { + Scene.v().getSootClassUnsafe(reflectClassName, false) + } catch (e: Exception) { + logErr("insertInstantStmt exception,reflectClassName is $reflectClassName") + return reflectMap + } + + if (reflectClass == null) { + return reflectMap + } + + val defaultMethodSubSig1 = "$reflectClassName getDefault()" + val defaultMethodSubSig2 = "$reflectClassName getInstance()" + val retName = "\$r" + ++localCnt + val ret = Jimple.v().newLocal(retName, reflectClass.type) as JimpleLocal + entryMethod.activeBody.locals.add(ret) + + val assignNewInstance: AssignStmt + if (reflectClass.declaresMethod(defaultMethodSubSig1) || reflectClass.declaresMethod(defaultMethodSubSig2)) { + var customInvoke = reflectClass.getMethodUnsafe(defaultMethodSubSig1) + if (customInvoke == null) { + customInvoke = reflectClass.getMethodUnsafe(defaultMethodSubSig2) + } + val staticInvokeExpr = Jimple.v().newStaticInvokeExpr(customInvoke!!.makeRef()) + assignNewInstance = Jimple.v().newAssignStmt(ret, staticInvokeExpr) + } else { + val newExpr = Jimple.v().newNewExpr(reflectClass.type) + assignNewInstance = Jimple.v().newAssignStmt(ret, newExpr) + } + patchUnits.add(assignNewInstance) + val invokeBaseLocal = forNameStmt.leftOp as JimpleLocal + val pair = Pair(ret, reflectClass) + reflectMap[invokeBaseLocal] = pair + + return reflectMap + } + + private fun insertGetClassInstantStmt( + forNameStmt: Stmt, + entryMethod: SootMethod, + patchUnits: MutableList + ): Map> { + var localCnt = entryMethod.activeBody.localCount + val reflectMap: MutableMap> = HashMap() + + if (forNameStmt !is JAssignStmt) { + return reflectMap + } + val forNameInvoke = forNameStmt.invokeExpr + val instanceInvokeExpr = forNameInvoke as InstanceInvokeExpr + val baseVal = instanceInvokeExpr.base //r3., + entryMethod: SootMethod, + patchUnits: MutableList + ): Map> { + var localCnt = entryMethod.activeBody.localCount + val reflectMap: MutableMap> = HashMap() + + for (forNameStmt in stmtSet) { + if (!forNameStmt.containsInvokeExpr()) { //invokeExpr instanceof InstanceInvokeExpr + continue + } + val invokeExpr = forNameStmt.invokeExpr + val invokeMethodSig = invokeExpr.methodRef.signature + if (invokeMethodSig != INVOKE_METHOD_SIG) { + continue + } + val arg1 = invokeExpr.getArg(0) + if (!arg1.toString().startsWith("class")) { + continue + } + val classNameArg = arg1.toString() // class "Landroid/telephony/SmsManager;" + + val reflectClassName = classNameArg.substring(8, classNameArg.length - 2) + .replace("/", ".") + val reflectClass: SootClass? = try { + Scene.v().getSootClassUnsafe(reflectClassName, false) + } catch (e: Exception) { + logErr("insertClassInstantStmt exception,reflectClassName is $reflectClassName") + continue + } + if (reflectClass == null) { + continue + } + val forNameInvoke = forNameStmt.invokeExpr + val instanceInvokeExpr = forNameInvoke as InstanceInvokeExpr + val baseVal = instanceInvokeExpr.base + val stmtReflectMethod = getStmt(entryMethod.toString(), baseVal.toString()) + ?: continue + val invokeTmp = stmtReflectMethod.invokeExpr as? InstanceInvokeExpr ?: continue + val baseFinal = invokeTmp.base + + val defaultMethodSubSig1 = "$reflectClassName getDefault()" + val defaultMethodSubSig2 = "$reflectClassName getInstance()" + val retName = "\$r" + ++localCnt + val ret = Jimple.v().newLocal(retName, reflectClass.type) as JimpleLocal + entryMethod.activeBody.locals.add(ret) + + var assignNewInstance: AssignStmt + if (reflectClass.declaresMethod(defaultMethodSubSig1) + || reflectClass.declaresMethod(defaultMethodSubSig2) + ) { + var customInvoke = reflectClass.getMethodUnsafe(defaultMethodSubSig1) + if (customInvoke == null) { + customInvoke = reflectClass.getMethodUnsafe(defaultMethodSubSig2) + } + val staticInvokeExpr = Jimple.v().newStaticInvokeExpr(customInvoke!!.makeRef()) + assignNewInstance = Jimple.v().newAssignStmt(ret, staticInvokeExpr) + } else { + val newExpr = Jimple.v().newNewExpr(reflectClass.type) + assignNewInstance = Jimple.v().newAssignStmt(ret, newExpr) + } +// entryMethod.activeBody.units.insertAfter(assignNewInstance, stmtReflectMethod) + patchUnits.add(assignNewInstance) + val invokeBaseLocal = baseFinal as JimpleLocal + val pair = Pair(ret, reflectClass) + reflectMap[invokeBaseLocal] = pair + } + return reflectMap + } + + private fun insertxxClassInstantStmt( + forNameStmt: Stmt, + entryMethod: SootMethod + ): Map> { + var localCnt = entryMethod.activeBody.localCount + val reflectMap: MutableMap> = HashMap() + if (!forNameStmt.containsInvokeExpr()) { + return reflectMap + } + val invokeExpr = forNameStmt.invokeExpr + val invokeMethodSig = invokeExpr.methodRef.signature + if (!invokeMethodSig.equals(INVOKE_METHOD_SIG)) { + return reflectMap + } + val arg1 = invokeExpr.getArg(0) + invokeExpr.getArg(1) + + val classNameArg = arg1.toString() + val reflectClassName: String + if (classNameArg.startsWith("class")) { + reflectClassName = classNameArg.substring(8, classNameArg.length - 2).replace("/", ".") + } else { + reflectClassName = getReflectionClass(entryMethod.toString(), arg1.toString()) + } + val reflectClass: SootClass? + if (reflectClassName.isEmpty()) { + return reflectMap + } + try { + reflectClass = Scene.v().getSootClassUnsafe(reflectClassName, false) + } catch (e: Exception) { + logErr("insertxxClassInstantStmt exception,reflectClassName is " + reflectClassName) + return reflectMap + } + if (reflectClass == null) { + return reflectMap + } + val forNameInvoke = forNameStmt.invokeExpr + val instanceInvokeExpr = forNameInvoke as InstanceInvokeExpr + val baseVal = instanceInvokeExpr.base + val stmtReflectMethod = getStmt(entryMethod.toString(), baseVal.toString()) + if (stmtReflectMethod == null || !stmtReflectMethod.containsInvokeExpr()) { + return reflectMap + } + val invokeTmp = stmtReflectMethod.invokeExpr + if (invokeTmp !is InstanceInvokeExpr) { + return reflectMap + } + val baseFinal = invokeTmp.base + + val defaultMethodSubSig1 = "$reflectClassName getDefault()" + val defaultMethodSubSig2 = "$reflectClassName getInstance()" + val retName = "\$r" + (++localCnt) + val ret = Jimple.v().newLocal(retName, reflectClass.type) as JimpleLocal + entryMethod.activeBody.locals.add(ret) + + val assignNewInstance: AssignStmt + if (reflectClass.declaresMethod(defaultMethodSubSig1) || reflectClass.declaresMethod(defaultMethodSubSig2)) { + var customInvoke = reflectClass.getMethodUnsafe(defaultMethodSubSig1) + if (customInvoke == null) { + customInvoke = reflectClass.getMethodUnsafe(defaultMethodSubSig2) + } + val staticInvokeExpr = Jimple.v().newStaticInvokeExpr(customInvoke.makeRef()) + assignNewInstance = Jimple.v().newAssignStmt(ret, staticInvokeExpr) + } else { + val newExpr = Jimple.v().newNewExpr(reflectClass.getType()) + assignNewInstance = Jimple.v().newAssignStmt(ret, newExpr) + } + entryMethod.activeBody.units.insertAfter(assignNewInstance, stmtReflectMethod) + + val invokeBaseLocal = baseFinal as JimpleLocal + val pair = Pair(ret, reflectClass) + reflectMap[invokeBaseLocal] = pair + + return reflectMap + } + + private fun getReflectionClass(methodName: String, leftParam: String): String { + val sm = Scene.v().getMethod(methodName) + if (sm.hasActiveBody()) { + for (unit in sm.activeBody.units) { + val stmt = unit as Stmt + if (stmt is JAssignStmt) { + val right = stmt.rightOp + val left = stmt.leftOp + if (left.toString() == leftParam) { + if (right.toString() + .startsWith("(") + ) { + return right.toString().split(")").toTypedArray()[0].substring(1) + } + if (right is InvokeExpr) { + val reflectClass = resolveMethodException(right).toString() + if (reflectClass.contains(":")) { + return reflectClass.split(":").toTypedArray()[0].substring(1) + } + } + } + } + if (stmt is JIdentityStmt) { + val right = stmt.rightOp.toString() + val left = stmt.leftOp.toString() + if (left == leftParam && right.startsWith("@parameter")) { + return right.substring(13) + } + } + } + } + return "" + } + + private fun getRightValue(methodSig: String?, leftArg: String): Value? { // $r4[2] = "hello" + val sm = Scene.v().getMethod(methodSig) + if (sm != null) { + for (unit in sm.activeBody.units) { + val stmt = unit as Stmt + if (stmt is JAssignStmt) { + val leftExpr = stmt.leftOp.toString() + val rightExpr = stmt.rightOp + if (leftExpr == leftArg) { + return rightExpr + } + } + } + } + return null + } + + private fun getStmt(methodSig: String?, leftArg: String): Stmt? { // xx.class + val sm = Scene.v().getMethod(methodSig) + if (sm != null) { + for (unit in sm.activeBody.units) { + val stmt = unit as Stmt + if (stmt is JAssignStmt) { + val leftExpr = stmt.leftOp.toString() + if (leftExpr == leftArg) { + return stmt + } + } + } + } + return null + } + + fun isXxClassReflection(methodName: String?): Boolean { + val sm = Scene.v().getMethod(methodName) + if (sm.hasActiveBody()) { + for (unit in sm.activeBody.units) { + val stmt = unit as Stmt + if (stmt.containsInvokeExpr()) { + val invokeExpr = stmt.invokeExpr + val invokeSig = invokeExpr.methodRef.signature + if (invokeSig == REFLECT_FOR_NAME || invokeSig == REFLECT_GET_CLASS) { + return false + } + } + } + } + return true + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/result/OutputSecResults.kt b/src/main/kotlin/net/bytedance/security/app/result/OutputSecResults.kt new file mode 100644 index 0000000..f10d740 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/result/OutputSecResults.kt @@ -0,0 +1,215 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +@file:Suppress("PropertyName") + +package net.bytedance.security.app.result + +import kotlinx.serialization.Serializable +import net.bytedance.security.app.Log +import net.bytedance.security.app.PLUtils +import net.bytedance.security.app.PreAnalyzeContext +import net.bytedance.security.app.android.AndroidUtils +import net.bytedance.security.app.getConfig +import net.bytedance.security.app.result.model.* +import net.bytedance.security.app.util.Json +import net.bytedance.security.app.util.TaskQueue +import net.bytedance.security.app.util.profiler +import net.bytedance.security.app.util.uploadJsonResult +import kotlin.system.exitProcess + +@Serializable +class Results { + var AppInfo: AppInfo? = null + var SecurityInfo: MutableMap> = HashMap() + var ComplianceInfo: MutableMap> = HashMap() + var DeepLinkInfo: MutableMap>? = null + var HTTP_API: MutableList? = null + var JsBridgeInfo: MutableList? = null + var BasicInfo: BasicInfo? = null + var UsePermissions: Set? = null + var DefinePermissions: Map? = null + var Profile: String? = null +} + +/** + * report output + */ +@Suppress("unused") +object OutputSecResults { + + private var Results = Results() + private var BasicInfo = BasicInfo() + + private var DeepLinkInfo: MutableMap> = HashMap() + private var AppInfo = AppInfo() + + + var APIList: MutableList = ArrayList() + + var JsBridgeList: MutableList = ArrayList() + + var JSList: MutableList = ArrayList() + private var vulnerabilityItems = HashSet() + fun init() { + Results.AppInfo = AppInfo + Results.DeepLinkInfo = DeepLinkInfo + Results.HTTP_API = APIList + Results.JsBridgeInfo = JsBridgeList + Results.BasicInfo = BasicInfo + BasicInfo.JSNativeInterface = JSList + BasicInfo.ComponentsInfo = AndroidUtils.compoXmlMapByType + initAppInfo() + } + + fun setAppInfoOther(other: Map) { + if (AppInfo.otherInfo == null) { + AppInfo.otherInfo = HashMap() + } + for (e in other) { + AppInfo.otherInfo!![e.key] = e.value + } + } + + private fun initAppInfo() { + AppInfo.AppName = AndroidUtils.AppLabelName + AppInfo.PackageName = AndroidUtils.PackageName + AppInfo.versionName = AndroidUtils.VersionName + AppInfo.versionCode = AndroidUtils.VersionCode + AppInfo.min_sdk = AndroidUtils.MinSdk + AppInfo.target_sdk = AndroidUtils.TargetSdk + profiler.AppInfo = AppInfo + } + + private fun insertPerm() { + Results.UsePermissions = AndroidUtils.usePermissionSet + Results.DefinePermissions = AndroidUtils.permissionMap + } + + fun insertDeepLink(key: String, set: Set) { + if (set.isEmpty()) { + return + } + val s = DeepLinkInfo.computeIfAbsent(key) { HashSet() } + s.addAll(set) + } + + + private suspend fun addManifest(ctx: PreAnalyzeContext) { + val manifestTaskQueue = + TaskQueue>("manifest", getConfig().maxThread) { task, _ -> + val t = TraceTask(ctx) + t.addManifest(task.second, task.first) + } + val job = manifestTaskQueue.runTask() + for (vulnerabilityItem in this.vulnerabilityItems) { + val taintPath = vulnerabilityItem.data.target + if (taintPath.isEmpty()) { + continue + } + val sourceMethodSig = taintPath[0].split("->").toTypedArray()[0] + val pair = Pair(sourceMethodSig, vulnerabilityItem) + manifestTaskQueue.addTask(pair) + } + manifestTaskQueue.addTaskFinished() + job.join() + } + + /** + * 1. Two different rules, may generate the same source and sink, such as IntentRedirection1 IntentRedirection2, + * one is DirectMode,the other is SliceMode + */ + private fun removeDup() { + + } + + /** + * group the results by the category + */ + private fun groupResult() { + for (vulnerabilityItem in this.vulnerabilityItems) { + val ruleDesc = vulnerabilityItem.rule.desc + val category: String + val subCategory: String = vulnerabilityItem.rule.desc.name + val m: MutableMap> + if (vulnerabilityItem.isCompliance()) { + category = ruleDesc.complianceCategory ?: "unknown" + m = Results.ComplianceInfo + } else { + category = ruleDesc.category + m = Results.SecurityInfo + } + val categoryMap = m.computeIfAbsent(category) { HashMap() } + val item = categoryMap.computeIfAbsent(subCategory) { + SecurityRiskItem( + ruleDesc.category, + ruleDesc.detail, + ruleDesc.model, + ruleDesc.name, + ruleDesc.possibility, + ArrayList(), + ruleDesc.wiki, + getConfig().deobfApk, + ruleDesc.level + ) + } + item.vulnerabilityItemMutableList.add(vulnerabilityItem.toSecurityVulnerabilityItem()) + } + } + + /** + * Add all the added information. The final report is the Results field + */ + suspend fun processResult(ctx: PreAnalyzeContext) { + try { + init() + insertPerm() + addManifest(ctx) + removeDup() + groupResult() + val jsonName = + "results_" + AndroidUtils.PackageName + "_" + java.lang.Long.toHexString(System.nanoTime() + (Math.random() * 100).toLong()) + val outputPath = getConfig().outPath + "/results.json" + val profileOutputPath = getConfig().outPath + "/profile.json" + profiler.processResult(Results) + Results.Profile = profiler.finishAndSaveProfilerResult() + val s = Json.encodeToPrettyString(Results) + PLUtils.writeFile(outputPath, s) + PLUtils.writeFile(profileOutputPath, profiler.toString()) + Log.logErr("write json to $outputPath") + uploadJsonResult("$jsonName.json", s) + } catch (ex: Exception) { + ex.printStackTrace() + exitProcess(21) + } + + } + + @Synchronized + fun addOneVulnerability(vulnerabilityItem: VulnerabilityItem) { + this.vulnerabilityItems.add(vulnerabilityItem) + } + + fun vulnerabilityItems(): Set { + return this.vulnerabilityItems + } + + //for test only + fun testClearVulnerabilityItems() { + this.vulnerabilityItems.clear() + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/result/TraceTask.kt b/src/main/kotlin/net/bytedance/security/app/result/TraceTask.kt new file mode 100644 index 0000000..fafdc3e --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/result/TraceTask.kt @@ -0,0 +1,119 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.result + +import net.bytedance.security.app.Log +import net.bytedance.security.app.PreAnalyzeContext +import net.bytedance.security.app.android.AndroidUtils +import net.bytedance.security.app.android.ComponentDescription +import net.bytedance.security.app.getConfig +import net.bytedance.security.app.ui.TaintPathModeVulnerability +import soot.Scene +import soot.SootClass +import soot.SootMethod +import java.util.* + +/** +add manifest field for vulnerability + */ +class TraceTask( + val ctx: PreAnalyzeContext, +) { + private fun getValidManifestEntry( + entryMethod: SootMethod, + validClassSet: MutableSet + ): ComponentDescription? { + + val sc = entryMethod.declaringClass + if (sc != null && !validClassSet.contains(sc)) { + validClassSet.add(sc) + val className = sc.name + return AndroidUtils.GlobalCompoXmlMap[className] + } + + return null + } + + /** + * add manifest for vulnerability + */ + fun addManifest(vulnerabilityItem: VulnerabilityItem, sourceMethodSig: String) { + val sourceMethod = Scene.v().getMethod(sourceMethodSig) + var componentJsonObj: ComponentDescription? = null + var entryClassName: String? = null + if (vulnerabilityItem.data !is TaintPathModeVulnerability) { + return + } + val entryMethod = vulnerabilityItem.data.entryMethod + if (AndroidUtils.entryCompoMap.containsKey(entryMethod)) { + val entryClass = AndroidUtils.entryCompoMap[entryMethod] + Log.logDebug(" class $entryClass") + entryClassName = entryClass!!.name + if (AndroidUtils.GlobalCompoXmlMap.containsKey(entryClassName)) { + componentJsonObj = AndroidUtils.GlobalCompoXmlMap[entryClassName] + } + } + + if (componentJsonObj != null) { + Log.logDebug("find direct entry ") + val componentJsonObj2 = componentJsonObj.clone() + + val path = ctx.callGraph.queryPath(entryMethod, sourceMethod, 16).map { it.signature }.toMutableList() + if (path.size > 0) { + path.removeAt(0) //Using real component names instead of our own constructed virtual entry method + } + path.add(0, entryClassName) + componentJsonObj2.trace = path + Log.logDebug("addTrace detailsJsonMap=$vulnerabilityItem,sourceMethodSig=$sourceMethodSig") + vulnerabilityItem.data.addManifest(componentJsonObj2) + Log.logDebug("results $vulnerabilityItem\n") + } else { + val validClassSet: MutableSet = HashSet() + val entryCallerSet: MutableSet = HashSet() + ctx.callGraph.queryTopEntryNoCustomMain( + false, + sourceMethod, + getConfig().manifestTrace * 3 / 2, + entryCallerSet + ) + entryCallerSet.add(sourceMethod) + val manifestList = LinkedList() + for (method in entryCallerSet) { + Log.logDebug("Entry $method") + val manifest2 = getValidManifestEntry(method, validClassSet) + if (manifest2 != null) { + val manifest = manifest2.clone() + val path = ctx.callGraph.queryPath(method, sourceMethod, getConfig().manifestTrace) + .map { it.signature } + if (path.isNotEmpty()) { + manifest.trace = path + if (manifest.exported) { + manifestList.addFirst(manifest) + } else { + manifestList.addLast(manifest) + } + } + } + } + for (jsonObject in manifestList) { + vulnerabilityItem.data.addManifest(jsonObject) + } + } + Log.logDebug("results $vulnerabilityItem\n") + } +} diff --git a/src/main/kotlin/net/bytedance/security/app/result/VulnerabilityItem.kt b/src/main/kotlin/net/bytedance/security/app/result/VulnerabilityItem.kt new file mode 100644 index 0000000..a4c61fe --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/result/VulnerabilityItem.kt @@ -0,0 +1,71 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.result + +import kotlinx.serialization.Serializable +import net.bytedance.security.app.result.model.AnySerializer +import net.bytedance.security.app.result.model.SecurityVulnerabilityItem +import net.bytedance.security.app.rules.IRule +import net.bytedance.security.app.util.Json +import org.apache.commons.codec.digest.DigestUtils + +interface IVulnerability { + fun toDetail(): Map + val target: List + val position: String +} + + +@Serializable +class VulnerabilityItem(val rule: IRule, val url: String, val data: IVulnerability) { + override fun hashCode(): Int { + return rule.hashCode() + url.hashCode() + data.hashCode() + } + + override fun equals(other: Any?): Boolean { + if (other is VulnerabilityItem) { + return other.rule == rule && other.url == url && other.data == data + } + return false + } + + override fun toString(): String { + return "vulnerabilityItem{name:${rule.name},url:$url,data:$data}" + } + + fun isCompliance(): Boolean { + return rule.desc.category == "ComplianceInfo" + } + + fun toSecurityVulnerabilityItem(): SecurityVulnerabilityItem { + val s = SecurityVulnerabilityItem() + s.possibility = rule.desc.possibility + val detail: MutableMap = HashMap() + detail.putAll(data.toDetail()) + s.details = detail + var data = Json.encodeToString(s) + data += rule.name + val hash = DigestUtils.sha1Hex(data) + if (url.isNotEmpty()) { + detail["url"] = url + } + s.hash = hash + s.details = detail + return s + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/result/model/result.kt b/src/main/kotlin/net/bytedance/security/app/result/model/result.kt new file mode 100644 index 0000000..b8ca3c3 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/result/model/result.kt @@ -0,0 +1,212 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +@file:Suppress("unused") + +package net.bytedance.security.app.result.model + +import kotlinx.serialization.KSerializer +import kotlinx.serialization.SerialName +import kotlinx.serialization.Serializable +import kotlinx.serialization.descriptors.SerialDescriptor +import kotlinx.serialization.descriptors.buildClassSerialDescriptor +import kotlinx.serialization.encoding.Decoder +import kotlinx.serialization.encoding.Encoder +import kotlinx.serialization.json.* +import net.bytedance.security.app.android.ComponentDescription +import net.bytedance.security.app.android.ToMapSerializeHelper +import kotlin.reflect.full.memberProperties + + +@Serializable +data class BasicInfo( + var AppInfo: AppInfo? = null, + var ComponentsInfo: MutableMap>? = null, + var PermissionInfo: MutableList? = null, + var SignInfo: SignInfo? = null, + var JSNativeInterface: List? = null +) + +@Serializable +class HttpAPI { + var API: String = "" + var AnnotType: MutableList = ArrayList() + var EntryMethod: String = "" + var ReqMethod: String = "" + + fun isEmpty(): Boolean { + return API.isEmpty() && AnnotType.isEmpty() && EntryMethod.isEmpty() && ReqMethod.isEmpty() + } +} + +@Serializable +class JsBridgeAPI { + var method: String = "" + var priv: String = "" + var sync: String = "" + var value: String = "" + + fun isEmpty(): Boolean { + return method.isEmpty() + } +} + +@Serializable +class AppInfo( + var AppName: String? = null, + var PackageName: String? = null, + var max_sdk: Int? = null, + var min_sdk: Int? = null, + var name: String? = null, + var sha1: String? = null, + var size: String? = null, + var target_sdk: Int? = null, + var versionCode: Int? = null, + var versionName: String? = null, + var otherInfo: MutableMap? = null +) + +@Serializable +data class ComponentsInfo( + var exportedActivities: MutableList, + var exportedProviders: MutableList, + var exportedReceivers: MutableList, + var exportedServices: MutableList, + var unExportedActivities: MutableList, + var unExportedProviders: MutableList, + var unExportedReceivers: MutableList, + var unExportedServices: MutableList +) + +@Serializable +data class SignInfo( + @SerialName("Is signed v1") var isSignedV1: Boolean, + @SerialName("Is signed v2") var isSignedV2: Boolean, + @SerialName("Is signed v3") var isSignedV3: Boolean, + var certs: MutableList, + var pkeys: MutableList +) + +@Serializable +data class Cert( + @SerialName("Hash Algorithm") var HashAlgorithm: String, + var Issuer: String, + @SerialName("Serial Number") var SerialNumber: String, + @SerialName("Signature Algorithm") var SignatureAlgorithm: String, + var Subject: String, + @SerialName("Valid not after") var ValidNotAfter: String, + @SerialName("Valid not before") var ValidNotBefore: String +) + +@Serializable +class SecurityRiskItem( + var category: String? = null, + var detail: String? = null, + var model: String? = null, + var name: String? = null, + var possibility: String? = null, + @SerialName("vulners") + var vulnerabilityItemMutableList: MutableList, + var wiki: String? = null, + var deobfApk: String? = null, + var level: String? = null, +) + +@Serializable +data class SecurityVulnerabilityDetail( + var Sink: MutableList?, + var Source: MutableList?, + var position: String?, + var target: MutableList?, + var url: String?, + var entryMethod: String?, + var others: MutableMap? +) + +@Serializable +data class SecurityVulnerabilityItem( + var details: Map? = null, + var hash: String? = null, + var possibility: String? = null +) + +object AnySerializer : KSerializer { + override val descriptor: SerialDescriptor = buildClassSerialDescriptor("Any") + + override fun serialize(encoder: Encoder, value: Any) { + val jsonEncoder = encoder as JsonEncoder + val jsonElement = serializeAny(value) + jsonEncoder.encodeJsonElement(jsonElement) + } + + private fun serializeAny(value: Any?): JsonElement = when (value) { + null -> JsonNull + is Map<*, *> -> { + val mapContents = value.entries.associate { mapEntry -> + mapEntry.key.toString() to serializeAny(mapEntry.value) + } + JsonObject(mapContents) + } + is List<*> -> { + val arrayContents = value.map { listEntry -> serializeAny(listEntry) } + JsonArray(arrayContents) + } + is Set<*> -> { + val arrayContents = value.map { listEntry -> serializeAny(listEntry) } + JsonArray(arrayContents) + } + is Number -> JsonPrimitive(value) + is Boolean -> JsonPrimitive(value) + is String -> JsonPrimitive(value) + else -> { + if (value is ToMapSerializeHelper) { + val mapContents = value.toMap().entries.associate { mapEntry -> + mapEntry.key to serializeAny(mapEntry.value) + } + JsonObject(mapContents) + } else { + val contents = value::class.memberProperties.associate { property -> + val v = try { + property.getter.call(value) + } catch (ex: Exception) { + ex.printStackTrace() + null + } + property.name to serializeAny(v) + } + JsonObject(contents) + } + } + } + + override fun deserialize(decoder: Decoder): Any { + val jsonDecoder = decoder as JsonDecoder + val element = jsonDecoder.decodeJsonElement() + + return deserializeJsonElement(element) + } + + private fun deserializeJsonElement(element: JsonElement): Any = when (element) { + is JsonObject -> { + element.mapValues { deserializeJsonElement(it.value) } + } + is JsonArray -> { + element.map { deserializeJsonElement(it) } + } + is JsonPrimitive -> element.toString() + } +} diff --git a/src/main/kotlin/net/bytedance/security/app/result/model/serializeAny.kt b/src/main/kotlin/net/bytedance/security/app/result/model/serializeAny.kt new file mode 100644 index 0000000..95c9a12 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/result/model/serializeAny.kt @@ -0,0 +1,39 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +@file:Suppress("unused") + +package net.bytedance.security.app.result.model + +import kotlinx.serialization.encodeToString +import kotlinx.serialization.json.* +import kotlinx.serialization.serializer +import kotlin.reflect.full.createType + +fun Any?.toJsonElement(): JsonElement = when (this) { + null -> JsonNull + is JsonElement -> this + is Number -> JsonPrimitive(this) + is Boolean -> JsonPrimitive(this) + is String -> JsonPrimitive(this) + is Array<*> -> JsonArray(map { it.toJsonElement() }) + is List<*> -> JsonArray(map { it.toJsonElement() }) + is Map<*, *> -> JsonObject(map { it.key.toString() to it.value.toJsonElement() }.toMap()) + else -> Json.encodeToJsonElement(serializer(this::class.createType()), this) +} + +fun Any?.toJsonString(): String = Json.encodeToString(this.toJsonElement()) diff --git a/src/main/kotlin/net/bytedance/security/app/ruleprocessor/ConstModeProcessor.kt b/src/main/kotlin/net/bytedance/security/app/ruleprocessor/ConstModeProcessor.kt new file mode 100644 index 0000000..06b8c98 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/ruleprocessor/ConstModeProcessor.kt @@ -0,0 +1,244 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.ruleprocessor + +import net.bytedance.security.app.MethodFinder +import net.bytedance.security.app.PLUtils +import net.bytedance.security.app.PreAnalyzeContext +import net.bytedance.security.app.SinkBody +import net.bytedance.security.app.preprocess.CallSite +import net.bytedance.security.app.result.OutputSecResults +import net.bytedance.security.app.rules.ConstNumberModeRule +import net.bytedance.security.app.rules.ConstStringModeRule +import net.bytedance.security.app.rules.TaintFlowRule +import net.bytedance.security.app.rules.TaintPosition +import net.bytedance.security.app.taintflow.TaintAnalyzer +import net.bytedance.security.app.ui.ConstExtractModeHtmlWriter +import soot.SootMethod +import soot.Value +import soot.jimple.* +import soot.jimple.internal.JimpleLocal + +abstract class ConstModeProcessor(ctx: PreAnalyzeContext) : TaintFlowRuleProcessor(ctx) { + suspend fun calcConstValueEntries( + rule: TaintFlowRule, + sink: Map, + analyzers: MutableList + ) { + + for ((methodSigRule, sinkContentObj) in sink) { + val methodSigSet = MethodFinder.checkAndParseMethodSig(methodSigRule) + if (sinkContentObj.TaintCheck == null) { + throw Exception("${rule.name} sink TaintCheck is null") + } + + val taintArr = sinkContentObj.TaintCheck + val taintParamTypeArr = sinkContentObj.TaintParamType + for (sinkMethodSig in methodSigSet) { + if (sinkContentObj.LibraryOnly == true && ctx.callGraph.isUserCode(sinkMethodSig)) { + continue + } + + val callSites = ctx.findInvokeCallSite(sinkMethodSig) + calcConstValFlowEntries(rule, callSites, taintArr, taintParamTypeArr, analyzers) + } + } + } + + + private suspend fun calcConstValFlowEntries( + rule: TaintFlowRule, + entry: Set, + paramArr: List?, + taintParamTypeArr: List?, + analyzers: MutableList + ) { + + entry.forEach { + val callerMethod = it.method + val callerStmtSet = it.stmt + val entrySet: MutableSet = HashSet() + ctx.callGraph.queryTopEntry(false, callerMethod, 1, entrySet) + if (entrySet.size > 1) { + entrySet.remove(callerMethod) + } + for (method in entrySet) { + val analyzer = TaintAnalyzer(rule, method) + calcAllConstPointers(rule, paramArr, taintParamTypeArr, callerMethod, setOf(callerStmtSet), analyzer) + if (analyzer.data.pointerIndexMap.isNotEmpty()) { + analyzers.add(analyzer) + } + } + } + } + + + private suspend fun calcAllConstPointers( + rule: TaintFlowRule, + paramArr: List?, + taintParamTypeArr: List?, + callerMethod: SootMethod, + callerStmtSet: Set, + analyzer: TaintAnalyzer + ) { + for (callerStmt in callerStmtSet) { + val invokeExpr = callerStmt.invokeExpr + if (paramArr != null) { + for (param in paramArr) { + val taintPosition = TaintPosition(param) + if (taintPosition.position == TaintPosition.AllArgument) { + for (arg in invokeExpr.args) { + if (!TaintRuleSourceSinkCollector.isValidType(taintParamTypeArr, arg.type)) { + continue + } + calcConstPtr(rule, arg, callerMethod, callerStmt, analyzer) + } + } else if (taintPosition.isConcreteArgument()) { + val index = taintPosition.position + if (index < invokeExpr.argCount) { + val arg = invokeExpr.getArg(index) + if (!TaintRuleSourceSinkCollector.isValidType(taintParamTypeArr, arg.type)) { + continue + } + calcConstPtr(rule, arg, callerMethod, callerStmt, analyzer) + } + } + } + } + } + } + + + private suspend fun calcConstPtr( + rule: TaintFlowRule, + arg: Value?, + callerMethod: SootMethod, + callerStmt: Stmt, + analyzer: TaintAnalyzer + ) { + if (arg is JimpleLocal) { + analyzer.data.allocPtrWithStmt(callerStmt, callerMethod, arg.name, arg.type, false) + } else if (arg is StringConstant && rule is ConstStringModeRule) { + constStrArgMatch(rule, callerMethod, callerStmt, arg.value) + } else if (arg is NumericConstant && rule is ConstNumberModeRule) { + constNumArgMatch(rule, callerMethod, callerStmt, arg) + } + } + + + suspend fun constStrArgMatch( + rule: ConstStringModeRule, + callerMethod: SootMethod, + callerStmt: Stmt, + constStr: String + ): Boolean { + var isMatch = false + if (rule.hasConstLen) { + val constStrLen = constStr.length + if (constStrLen > 0 && rule.constLen != null && constStrLen % rule.constLen == 0) { + + if (rule.targetStringArr != null) { + if (isMatchTargetConstStr(constStr, rule.targetStringArr)) { + isMatch = true + } + } else { + isMatch = true + } + } + } else { + if (rule.targetStringArr != null) { + if (isMatchTargetConstStr(constStr, rule.targetStringArr)) { + isMatch = true + } + } else if (rule.minLen != null) { + if (constStr.length >= rule.minLen) { + isMatch = true + } + } else { + isMatch = true + } + } + if (isMatch) { + ConstExtractModeHtmlWriter( + OutputSecResults, + rule, + callerMethod, + callerStmt, + constStr + ).addVulnerabilityAndSaveResultToOutput() + } + return isMatch + } + + + suspend fun constNumArgMatch( + rule: ConstNumberModeRule, + callerMethod: SootMethod, + callerStmt: Stmt, + arg: NumericConstant + ): Boolean { + if (arg is IntConstant) { + return constNumArgMatch(rule, callerMethod, callerStmt, arg.value.toLong()) + } else if (arg is LongConstant) { + return constNumArgMatch(rule, callerMethod, callerStmt, arg.value) + } + return false + } + + + suspend fun constNumArgMatch( + rule: ConstNumberModeRule, + callerMethod: SootMethod, + callerStmt: Stmt, + num: Long + ): Boolean { + if (rule.targetNumberArr != null) { + if (rule.targetNumberArr.contains(num.toInt())) { + ConstExtractModeHtmlWriter( + OutputSecResults, + rule, + callerMethod, + callerStmt, + num.toString() + ).addVulnerabilityAndSaveResultToOutput() + return true + } + } else { + ConstExtractModeHtmlWriter( + OutputSecResults, + rule, + callerMethod, + callerStmt, + num.toString() + ).addVulnerabilityAndSaveResultToOutput() + return true + } + return false + } + + companion object { + + fun isMatchTargetConstStr(targetStr: String, targetStrArr: List): Boolean { + for (str in targetStrArr) { + if (PLUtils.isStrMatch(str, targetStr)) + return true + } + return false + } + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/ruleprocessor/ConstNumberModeProcessor.kt b/src/main/kotlin/net/bytedance/security/app/ruleprocessor/ConstNumberModeProcessor.kt new file mode 100644 index 0000000..84b37db --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/ruleprocessor/ConstNumberModeProcessor.kt @@ -0,0 +1,47 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.ruleprocessor + +import net.bytedance.security.app.PreAnalyzeContext +import net.bytedance.security.app.rules.ConstNumberModeRule +import net.bytedance.security.app.rules.IRule +import net.bytedance.security.app.taintflow.TaintAnalyzer + +/** + * The difference from ConstStringMode is that the source is literal number + */ +class ConstNumberModeProcessor(ctx: PreAnalyzeContext) : + ConstModeProcessor(ctx) { + + override fun name(): String { + return "ConstNumberMode" + } + + override suspend fun process(rule: IRule) { + if (rule !is ConstNumberModeRule) { + return //panic? + } + val analyzers = ArrayList() + parseConstNumberMode(rule, analyzers) + this.collectAnalyzers(analyzers, rule) + } + + private suspend fun parseConstNumberMode(rule: ConstNumberModeRule, analyzers: MutableList) { + calcConstValueEntries(rule, rule.sink, analyzers) + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/ruleprocessor/ConstStringModeProcessor.kt b/src/main/kotlin/net/bytedance/security/app/ruleprocessor/ConstStringModeProcessor.kt new file mode 100644 index 0000000..c8279f0 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/ruleprocessor/ConstStringModeProcessor.kt @@ -0,0 +1,51 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.ruleprocessor + +import net.bytedance.security.app.Log +import net.bytedance.security.app.PreAnalyzeContext +import net.bytedance.security.app.rules.ConstStringModeRule +import net.bytedance.security.app.rules.IRule +import net.bytedance.security.app.taintflow.TaintAnalyzer + + +class ConstStringModeProcessor(ctx: PreAnalyzeContext) : ConstModeProcessor(ctx) { + + override fun name(): String { + return "ConstStringMode" + } + + override suspend fun process(rule: IRule) { + if (rule !is ConstStringModeRule) { + return //panic? + } + val analyzers = ArrayList() + parseConstStringMode(rule, analyzers) + this.collectAnalyzers(analyzers, rule) + } + + private suspend fun parseConstStringMode( + rule: ConstStringModeRule, + analyzers: MutableList + ) { + Log.logDebug("\tConstStringMode enabled") + calcConstValueEntries(rule, rule.sink, analyzers) + } + + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/ruleprocessor/DirectModeProcessor.kt b/src/main/kotlin/net/bytedance/security/app/ruleprocessor/DirectModeProcessor.kt new file mode 100644 index 0000000..221d0f9 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/ruleprocessor/DirectModeProcessor.kt @@ -0,0 +1,139 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.ruleprocessor + +import net.bytedance.security.app.* +import net.bytedance.security.app.Log.logInfo +import net.bytedance.security.app.android.AndroidUtils +import net.bytedance.security.app.engineconfig.isLibraryClass +import net.bytedance.security.app.rules.DirectModeRule +import net.bytedance.security.app.rules.IRule +import net.bytedance.security.app.taintflow.TaintAnalyzer +import soot.Scene +import soot.SootMethod + + +open class DirectModeProcessor(ctx: PreAnalyzeContext) : TaintFlowRuleProcessor(ctx) { + override fun name(): String { + return "DirectMode" + } + + override suspend fun process(rule: IRule) { + if (rule !is DirectModeRule) { + return + } + logInfo("${this.javaClass.name} process rule ${rule.name}") + val entries = createEntries(rule) + val taintRuleSourceSinkCollector = TaintRuleSourceSinkCollector(ctx, rule, entries) + taintRuleSourceSinkCollector.collectSourceSinks() + val analyzers = ArrayList() + createAnalyzers(rule, taintRuleSourceSinkCollector, entries, analyzers) + this.collectAnalyzers(analyzers, rule) + } + + open suspend fun createEntries(rule: DirectModeRule): List { + val entries: Set = + if (getConfig().doWholeProcessMode) { + setOf(Scene.v().grabMethod(PLUtils.CUSTOM_CLASS_ENTRY)) + } else { + val s = HashSet() + parseDirectEntry(rule.entry, s) + s + } + return entries.toList() + } + + open suspend fun createAnalyzers( + rule: DirectModeRule, + taintRuleSourceSinkCollector: TaintRuleSourceSinkCollector, + entries: List, + analyzers: MutableList + ) { + for (entry in entries) { + if (!taintRuleSourceSinkCollector.entryHasValidSource(entry)) { + continue + } + val analyzer = TaintAnalyzer(rule, entry, taintRuleSourceSinkCollector.analyzerData) + analyzers.add(analyzer) + } + + } + + /* + entry: "entry": { + "methods": [ + "<*: android.webkit.WebResourceResponse shouldInterceptRequest(android.webkit.WebView,android.webkit.WebResourceRequest)>", + "<*: android.webkit.WebResourceResponse shouldInterceptRequest(android.webkit.WebView,java.lang.String)>" + ], + "components": [] + } + */ + private fun parseDirectEntry(entry: Entry?, entries: MutableSet) { + if (entry == null) { + return + } + if (entry.ExportedCompos == true) { + for (exportCompo in AndroidUtils.exportComponents) { + if (AndroidUtils.compoEntryMap.containsKey(exportCompo)) { + val sootMethod = AndroidUtils.compoEntryMap[exportCompo] + entries.add(sootMethod!!) + } + } + } + if (entry.UseJSInterface == true) { + jsInterfaceAsEntry(entries) + } + + val components = entry.components + if (components != null) { + for (className in components) { + val sc = Scene.v().getSootClassUnsafe(className, false) + if (AndroidUtils.compoEntryMap.containsKey(sc)) { + val sootMethod = AndroidUtils.compoEntryMap[sc] + entries.add(sootMethod!!) + } + } + } + val entryMethods = entry.methods + if (entryMethods != null) { + for (entryMethodSig in entryMethods) { + val methodSet = MethodFinder.checkAndParseMethodSig(entryMethodSig) + for (method in methodSet) { + if (isLibraryClass(method.declaringClass.name)) { + continue + } + if (method.isConcrete) { + entries.add(method) + } + } + } + } + + Log.logDebug("Direct Entry Size " + entries.size) + } + + private fun jsInterfaceAsEntry(entries: MutableSet) { + if (ctx !is ContextWithJSBMethods) { + return + } + for (sm in ctx.getJSBMethods()) { + entries.add(sm) + } + } + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/ruleprocessor/IRuleProcessor.kt b/src/main/kotlin/net/bytedance/security/app/ruleprocessor/IRuleProcessor.kt new file mode 100644 index 0000000..e357ac6 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/ruleprocessor/IRuleProcessor.kt @@ -0,0 +1,25 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.ruleprocessor + +import net.bytedance.security.app.rules.IRule + +interface IRuleProcessor { + fun name(): String + suspend fun process(rule: IRule) +} diff --git a/src/main/kotlin/net/bytedance/security/app/ruleprocessor/RuleProcessorFactory.kt b/src/main/kotlin/net/bytedance/security/app/ruleprocessor/RuleProcessorFactory.kt new file mode 100644 index 0000000..9a024d9 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/ruleprocessor/RuleProcessorFactory.kt @@ -0,0 +1,38 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.ruleprocessor + +import net.bytedance.security.app.PreAnalyzeContext + +object RuleProcessorFactory { + fun create(ctx: PreAnalyzeContext, mode: String): IRuleProcessor { + + val p = when (mode) { + "ConstStringMode" -> ConstStringModeProcessor(ctx) + "ConstNumberMode" -> ConstNumberModeProcessor(ctx) + "DirectMode" -> DirectModeProcessor(ctx) + "SliceMode" -> SliceModeProcessor(ctx) + else -> throw Exception("Unknown mode: $mode") + } + if (p.name() == "DirectMode") { + assert(p is DirectModeProcessor) + } + assert(p.name() == mode) + return p + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/ruleprocessor/SliceModeProcessor.kt b/src/main/kotlin/net/bytedance/security/app/ruleprocessor/SliceModeProcessor.kt new file mode 100644 index 0000000..403590c --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/ruleprocessor/SliceModeProcessor.kt @@ -0,0 +1,103 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.ruleprocessor + +import kotlinx.coroutines.* +import net.bytedance.security.app.PreAnalyzeContext +import net.bytedance.security.app.getConfig +import net.bytedance.security.app.rules.DirectModeRule +import net.bytedance.security.app.rules.SliceModeRule +import net.bytedance.security.app.taintflow.TaintAnalyzer +import soot.SootMethod + + +class SliceModeProcessor(ctx: PreAnalyzeContext) : DirectModeProcessor(ctx) { + + override fun name(): String { + return "SliceMode" + } + + + override suspend fun createAnalyzers( + rule: DirectModeRule, + taintRuleSourceSinkCollector: TaintRuleSourceSinkCollector, + entries: List, + analyzers: MutableList + ) { + assert(rule is SliceModeRule && rule.isSliceEnable) + if (!getConfig().doWholeProcessMode) { + createAnalyzersForSourceAndSink(taintRuleSourceSinkCollector, rule as SliceModeRule, analyzers) + } else { + super.createAnalyzers(rule, taintRuleSourceSinkCollector, entries, analyzers) + } + } + + suspend fun createAnalyzersForSourceAndSink( + taintRuleSourceSinkCollector: TaintRuleSourceSinkCollector, + rule: SliceModeRule, + analyzers: MutableList + ) { + val jobs = ArrayList() + val scope = CoroutineScope(Dispatchers.Default) + for (srcPtr in taintRuleSourceSinkCollector.analyzerData.sourcePointerSet) { + //if srcPtr is a library method, it + val callstacks = if (taintRuleSourceSinkCollector.parameterSources.contains(srcPtr)) { + this.ctx.callGraph.getAllCallees(srcPtr.method, rule.traceDepth) + } else { + null + } + for (sinkPtr in taintRuleSourceSinkCollector.analyzerData.sinkPointerSet) { + val job = scope.launch(CoroutineName("createAnalyzersForSourceAndSink-${rule.name}")) { + + val entryItem = if (callstacks == null) { + val result = ctx.callGraph.traceAndCross( + rule.polymorphismBackTrace, + srcPtr.method, + sinkPtr.method, + rule.traceDepth - 2 + ) ?: return@launch + val thisDepth = rule.traceDepth - result.depth + Pair(result.entryMethod, thisDepth) + } else { + if (!callstacks.contains(sinkPtr.method)) { + return@launch + } + Pair(srcPtr.method, rule.traceDepth) + } + + val analyzer = TaintAnalyzer( + rule, + entryItem.first, + taintRuleSourceSinkCollector.analyzerData, + srcPtr, + sinkPtr, + entryItem.second + ) + synchronized(analyzers) { + analyzers.add(analyzer) + + } + } +// job.join() //for test only + jobs.add(job) + } + } + jobs.joinAll() + } + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/ruleprocessor/TaintFlowRuleProcessor.kt b/src/main/kotlin/net/bytedance/security/app/ruleprocessor/TaintFlowRuleProcessor.kt new file mode 100644 index 0000000..e721842 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/ruleprocessor/TaintFlowRuleProcessor.kt @@ -0,0 +1,42 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.ruleprocessor + +import net.bytedance.security.app.Log.logInfo +import net.bytedance.security.app.PreAnalyzeContext +import net.bytedance.security.app.rules.IRule +import net.bytedance.security.app.taintflow.TaintAnalyzer +import net.bytedance.security.app.util.profiler + +abstract class TaintFlowRuleProcessor(val ctx: PreAnalyzeContext) : IRuleProcessor { + val analyzers: MutableList = ArrayList() + + @Synchronized + fun collectAnalyzers(analyzers: List, rule: IRule) { + try { + analyzers.forEach { + this.analyzers.add(it) + } + logInfo("${rule.name} collected ${analyzers.size} analyzers") + } catch (ex: Exception) { + ex.printStackTrace() + } + profiler.setRuleAnalyzerCount(rule.name, analyzers.size) + } + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/ruleprocessor/TaintRuleSourceSinkCollector.kt b/src/main/kotlin/net/bytedance/security/app/ruleprocessor/TaintRuleSourceSinkCollector.kt new file mode 100644 index 0000000..7fa7648 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/ruleprocessor/TaintRuleSourceSinkCollector.kt @@ -0,0 +1,430 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.ruleprocessor + + +import net.bytedance.security.app.* +import net.bytedance.security.app.android.AndroidUtils +import net.bytedance.security.app.pointer.PLLocalPointer +import net.bytedance.security.app.preprocess.CallSite +import net.bytedance.security.app.rules.DirectModeRule +import net.bytedance.security.app.rules.TaintPosition +import net.bytedance.security.app.taintflow.TaintAnalyzerData +import net.bytedance.security.app.util.toSortedMap +import soot.* +import soot.jimple.InstanceInvokeExpr +import soot.jimple.Stmt +import soot.jimple.StringConstant +import soot.jimple.internal.JAssignStmt +import soot.jimple.internal.JimpleLocal + +class TaintRuleSourceSinkCollector( + val ctx: PreAnalyzeContext, + val rule: DirectModeRule, + entries: List, +) { + val analyzerData = TaintAnalyzerData() + + val parameterSources = HashSet() + val source = rule.source!! + val sink: Map = rule.sink + + //key is entry method + private val hasSourceReturn = HashMap() + + init { + entries.forEach { + hasSourceReturn[it] = false + } + } + + fun collectSourceSinks() { + processSource() + processSink() + } + + private fun processSink() { + for ((sinkKey, sinkContentObj) in toSortedMap(sink)) { + val sinkMethodSet = MethodFinder.checkAndParseMethodSig(sinkKey) + + if (sinkMethodSet.isEmpty()) { + continue + } + for (sinkMethodSig in sinkMethodSet) { + if (sinkContentObj.LibraryOnly == true && ctx.callGraph.isUserCode(sinkMethodSig)) { + continue + } + val sinkCallSites = ctx.findInvokeCallSite(sinkMethodSig) + if (sinkCallSites.isEmpty()) { + continue + } + findSinkPointersForOneSinkRule(sinkContentObj, sinkCallSites) + } + } + } + + private fun findSinkPointersForOneSinkRule( + sinkContentObj: SinkBody, + sinkCallSites: Set + ) { + for (callsite in sinkCallSites) { + calcSinksAndParamCheck( + sinkContentObj, + setOf(callsite.stmt), + callsite.method + ) + } + } + + private fun calcSinksAndParamCheck( + sinkContentObj: SinkBody, + sinkStmtSet: Set, + sinkMethodCaller: SootMethod + ) { + + for (sinkStmt in sinkStmtSet) { + calcSinkPointers( + sinkStmt, + sinkContentObj, + sinkMethodCaller + ) + + } + } + + + private fun calcSinkPointers( + stmt: Stmt, + sink: SinkBody, + sinkMethodCaller: SootMethod + ): Set { + if (sink.TaintCheck == null || sink.TaintCheck.isEmpty()) { + Log.logErr("${this.rule.name} sink TaintCheck is empty") + return emptySet() + } + val ptrSet: MutableSet = HashSet() + val invokeExpr = stmt.invokeExpr + val paramArr = sink.TaintCheck + val paramTypeArr = sink.TaintParamType + + for (checkParamStr in paramArr) { + val tp = TaintPosition(checkParamStr) + if (tp.position == TaintPosition.This) { + if (invokeExpr is InstanceInvokeExpr) { + val base = invokeExpr.base + val ptr = addPtrToEntry(stmt, base, sinkMethodCaller) + if (ptr != null) { + ptrSet.add(ptr) + } + } + } else if (tp.position == TaintPosition.AllArgument) { + for (arg in invokeExpr.args) { + if (!isValidType(paramTypeArr, arg.type)) { + continue + } + val ptr = addPtrToEntry(stmt, arg, sinkMethodCaller) + if (ptr != null) { + ptrSet.add(ptr) + } + } + } else if (tp.position >= 0) { + if (tp.position < invokeExpr.argCount) { + val arg = invokeExpr.getArg(tp.position) + if (!isValidType(paramTypeArr, arg.type)) { + continue + } + val ptr = addPtrToEntry(stmt, arg, sinkMethodCaller) + if (ptr != null) { + ptrSet.add(ptr) + } + } + } else { + throw Exception("return cannot be sink position for rule {${rule.name}") + } + } + return ptrSet + } + + private fun addPtrToEntry( + stmt: Stmt, + arg: Value, + callerMethod: SootMethod, + ): PLLocalPointer? { + if (arg is StringConstant) { + return analyzerData.allocPtrWithStmt( + stmt, + callerMethod, + PLUtils.constStrSig(arg.value), + RefType.v("java.lang.String"), false + ) + } else if (arg is JimpleLocal) { + return analyzerData.allocPtrWithStmt( + stmt, + callerMethod, + arg.name, + arg.getType(), false + ) + } + return null + } + + private fun processSource() { + if (source.ConstString.isNotEmpty()) { + processSourceConstStr(source.ConstString) + } + if (source.StaticField.isNotEmpty()) { + processSourceLoadField(source.StaticField) + } + if (source.Field.isNotEmpty()) { + processSourceLoadField(source.Field) + } + if (source.Return != null) { + processSourceReturn(source.parseReturn()) + } + if (source.Param.isNotEmpty()) { + processSourceMethodParameter(source.Param) + } + if (source.NewInstance.isNotEmpty()) { + processSourceNewInstance(source.NewInstance) + } + if (source.UseJSInterface) { + processSourceUseJSInterface() + } + + } + + fun entryHasValidSource(entry: SootMethod): Boolean { + if (source.ConstString.isNotEmpty() || source.StaticField.isNotEmpty() || source.Param.isNotEmpty() || source.NewInstance.isNotEmpty()) { + return true + } + if (source.Return != null && hasSourceReturn[entry] == false) { + return false + } + return true + } + + /** + * if a method is a jsb method, all arguments can be controlled by javascript. + */ + private fun processSourceUseJSInterface() { + if (ctx !is ContextWithJSBMethods) { + return + } + for (sm in ctx.getJSBMethods()) { + for (i in 0 until sm.parameterCount) { + val paramType = sm.getParameterType(i) + if (paramType is PrimType) { + continue + } + val ptr = analyzerData.allocSourcePtr(sm, PLUtils.PARAM + i, sm.getParameterType(i)) + parameterSources.add(ptr) + } + } + } + + private fun processSourceLoadField( + fields: List, + ) { + for (field in fields) { + val callsites = ctx.findFieldCallSite(field) + for (callsite in callsites) { + allocDirectEntrySourcePtr(callsite, callsite.method) + } + } + } + + private fun processSourceNewInstance( + jsonNewArray: List, + ) { + for (obj in jsonNewArray) { + val callsites = ctx.findInstantCallSiteWithSubclass(obj) + for (callsite in callsites) { + allocDirectEntrySourcePtr(callsite, callsite.method) + } + } + } + + private fun processSourceMethodParameter( + paramObj: Map> + ) { + for ((methodKey, parameters) in paramObj) { + val sourceMethodSet = MethodFinder.checkAndParseMethodSig(methodKey) + if (sourceMethodSet.isEmpty()) { + continue + } + for (sourceMethod in sourceMethodSet) { + Log.logDebug("source $sourceMethod") + for (param in parameters) { + val tp = TaintPosition(param) + if (tp.position == TaintPosition.AllArgument) { + for (i in 0 until sourceMethod.parameterCount) { + val ptr = analyzerData.allocSourcePtr( + sourceMethod, + PLUtils.PARAM + i, + sourceMethod.getParameterType(i), + ) + parameterSources.add(ptr) + } + } else if (tp.isConcreteArgument()) { + val index = tp.position + if (index < sourceMethod.parameterCount) { + val ptr = analyzerData.allocSourcePtr( + sourceMethod, + PLUtils.PARAM + index, + sourceMethod.getParameterType(index), + ) + parameterSources.add(ptr) + } + } else { + Log.logErr("source param position $param is not valid in ${rule.name}") + } + } + } + } + } + + private fun processSourceReturn( + returns: Map, + ) { + for ((methodSig, cfg) in returns) { + val sourceMethodSet = MethodFinder.checkAndParseMethodSig(methodSig) + for (source in sourceMethodSet) { + if (cfg.LibraryOnly == true && ctx.callGraph.isUserCode(source)) { + continue + } + + val callsites = ctx.findInvokeCallSite(source) + for (callsite in callsites) { + if (cfg.EntryInvoke && !getConfig().doWholeProcessMode) { + val sourceClass = + source.declaringClass + if (isMethodHasParent(source, sourceClass)) { + continue + } + if (!isSourceCalledInExportedComponents(callsite.method)) { + continue + } + } + if (!cfg.EntryInvoke) { + //if there is one source doesn't need EntryInvoke, then all entries are valid. + for ((k, _) in hasSourceReturn) { + hasSourceReturn[k] = true + } + } + allocDirectEntrySourcePtr(callsite, callsite.method) + } + } + } + } + + private fun isSourceCalledInExportedComponents(source: SootMethod): Boolean { + val sourceClass = source.declaringClass + var found = false + for (entryMethod in this.hasSourceReturn.keys) { + var entryClass = entryMethod.declaringClass + if (AndroidUtils.entryCompoMap.containsKey(entryMethod)) { + entryClass = AndroidUtils.entryCompoMap[entryMethod] + } + AndroidUtils.dummyToDirectEntryMap[entryMethod]?.let { + entryClass = it.declaringClass + } + if (!Scene.v().orMakeFastHierarchy.canStoreClass(entryClass, sourceClass)) { + continue + } + this.hasSourceReturn[entryMethod] = true + found = true + } + return found + + } + + private fun allocDirectEntrySourcePtr( + callsite: CallSite, + sourceMethod: SootMethod, + ) { + if (callsite.stmt is JAssignStmt) { + val local = callsite.stmt.leftOp as? JimpleLocal ?: return + analyzerData.allocPtrWithStmt(callsite.stmt, sourceMethod, local.name, local.type, true) + } + } + + + private fun processSourceConstStr( + jsonStrArray: List, + ) { + for (pattern in jsonStrArray) { + val constCallSites = ctx.findConstStringPatternCallSite(pattern) + for (callsite in constCallSites) { + + val constStrings = callsite.constString() + for (constString in constStrings) { + if (!PLUtils.isStrMatch(pattern, constString)) { + continue + } + analyzerData.allocPtrWithStmt( + callsite.stmt, + callsite.method, + PLUtils.constStrSig(constString), + RefType.v("java.lang.String"), true + ) + } + } + } + } + + + companion object { + /** + * is method override of library method + */ + private fun isMethodHasParent(method: SootMethod, declareClass: SootClass): Boolean { + val declareSubMethodSig = method.subSignature + val classSet: MutableSet = java.util.HashSet() + getAllSuperClass(declareClass, classSet) + for (parentClass in classSet) { + val parentMethod = parentClass.getMethodUnsafe(declareSubMethodSig) + if (parentMethod != null) { + return true + } + } + return false + } + + private fun getAllSuperClass(sc: SootClass, superClasses: MutableSet) { + if (sc.hasSuperclass()) { + val sootClass = sc.superclass + superClasses.add(sootClass) + // logErr(sc.getName()+"'s super is "+sootClass.getName()); + getAllSuperClass(sootClass, superClasses) + } + } + + fun isValidType(paramTypeArr: List?, type: Type): Boolean { + if (paramTypeArr == null) { + return true + } + for (typeStr in paramTypeArr) { + if (typeStr == type.toString()) { + return true + } + } + return false + } + } + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/rules/AbstractRule.kt b/src/main/kotlin/net/bytedance/security/app/rules/AbstractRule.kt new file mode 100644 index 0000000..37e8494 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/rules/AbstractRule.kt @@ -0,0 +1,29 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.rules + +import net.bytedance.security.app.RuleData +import net.bytedance.security.app.RuleDescription + +abstract class AbstractRule(override val name: String, ruleData: RuleData) : IRule { + final override val desc: RuleDescription + + init { + desc = ruleData.desc + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/rules/ConstNumberModeRule.kt b/src/main/kotlin/net/bytedance/security/app/rules/ConstNumberModeRule.kt new file mode 100644 index 0000000..29ffff9 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/rules/ConstNumberModeRule.kt @@ -0,0 +1,35 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.rules + +import net.bytedance.security.app.RuleData + + +class ConstNumberModeRule(name: String, ruleData: RuleData) : TaintFlowRule(name, ruleData) { + override val mode: String = "ConstNumberMode" +// +// val hasTargetNum: Boolean + + val targetNumberArr: List? + + + init { + targetNumberArr = ruleData.targetNumberArr + this.primTypeAsTaint = true + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/rules/ConstStringModeRule.kt b/src/main/kotlin/net/bytedance/security/app/rules/ConstStringModeRule.kt new file mode 100644 index 0000000..b92a1e1 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/rules/ConstStringModeRule.kt @@ -0,0 +1,41 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.rules + +import net.bytedance.security.app.RuleData + + +class ConstStringModeRule(name: String, ruleData: RuleData) : TaintFlowRule(name, ruleData), IRuleConstStringPattern { + override val mode: String = "ConstStringMode" + val constLen: Int? + val minLen: Int? + val targetStringArr: List? + val hasConstLen: Boolean + + + init { + constLen = ruleData.constLen + minLen = ruleData.minLen + targetStringArr = ruleData.targetStringArr + this.hasConstLen = constLen != null + } + + override fun constStringPatterns(): Set { + return targetStringArr?.toSet() ?: emptySet() + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/rules/DirectModeRule.kt b/src/main/kotlin/net/bytedance/security/app/rules/DirectModeRule.kt new file mode 100644 index 0000000..50c76bc --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/rules/DirectModeRule.kt @@ -0,0 +1,232 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.rules + +import kotlinx.serialization.json.JsonArray +import kotlinx.serialization.json.JsonObject +import kotlinx.serialization.json.jsonObject +import kotlinx.serialization.json.jsonPrimitive +import net.bytedance.security.app.* +import net.bytedance.security.app.util.Json +import net.bytedance.security.app.util.isFieldSignature + +open class DirectModeRule(name: String, ruleData: RuleData) : TaintFlowRule(name, ruleData), IRuleConstStringPattern, + IRuleNewInstance, + IRuleField { + override val mode: String = "DirectMode" + val entry: Entry? + val sinkRules: List? + val sourceRules: List? + val throughAPI: ThroughAPI? + + + private var isInitComplete = false + suspend fun initIfNeeded() { + if (!isInitComplete) { + val ss = processSourceAndSink() + this.source = ss.first + this.sink = ss.second + } + isInitComplete = true + } + + override fun constStringPatterns(): Set { + val constStrings = source?.ConstString?.let { + ArrayList(it) + } ?: ArrayList() + if (sanitize != null) { + val values = sanitize.values + for (m in values) { + for ((sig, sinkBodyElement) in m) { + if (sig == "ConstString") { + val strings: List = Json.decodeFromJsonElement(sinkBodyElement) + constStrings.addAll(strings) + continue + } + val sinkBody = Json.decodeFromJsonElement(sinkBodyElement) + if (sinkBody.pstar != null) { + val pstar = sinkBody.pstar + for (p in pstar) { + if (p.jsonPrimitive.isString) { + val s = p.jsonPrimitive.content + if (!s.startsWith("@")) { + constStrings.add(s) + } + } + } + } + if (sinkBody.pmap != null) { + for ((_, value) in sinkBody.pmap!!) { + for (p in value) { + if (p.jsonPrimitive.isString) { + val s = p.jsonPrimitive.content + if (!s.startsWith("@")) { + constStrings.add(s) + } + } + } + } + } + + } + } + } + return constStrings.toSet() + } + + override fun newInstances(): Set { + source?.NewInstance?.let { + return it.toSet() + } + return emptySet() + } + + override fun fields(): Set { + val results = source?.StaticField?.toMutableSet() ?: HashSet() + source?.Field?.let { + for (field in it) { + results.add(field) + } + } + if (sanitize != null) { + val values = sanitize.values + for (m in values) { + for ((key, _) in m) { + if (key.isFieldSignature()) { + results.add(key) + } + } + } + } + return results + } + + + private suspend fun processSourceAndSink(): Pair> { + val source = this.source ?: SourceBody() + var sink = this.sink + this.sourceRules?.forEach { + parseSourceRuleObj(it, source) + } + this.sinkRules?.forEach { + sink = parseSinkRuleObj(it, sink) + } + return Pair(source, sink) + } + + private suspend fun parseSourceRuleObj(ruleObj: RuleObjBody, source: SourceBody) { + source.Return = source.Return ?: JsonArray(emptyList()) + val secRules = loadRuleFromFile(ruleObj.ruleFile!!) + secRules.keys.forEach { ruleName -> + if (ruleObj.include.isNotEmpty() && !ruleObj.include.contains(ruleName)) { + return@forEach + } + val jsonObj = secRules[ruleName]!! + val ruleData: RuleData = Json.decodeFromJsonElement(jsonObj) + if (ruleObj.fromAPIMode) { + if (ruleData.APIMode == true) { + ruleData.sink?.run { + val filter1: (String) -> Boolean = { x -> x.contains("(") } + source.RuleObjReturn = listMerge(source.RuleObjReturn, this.keys.toList(), filter1) + val filter2: (String) -> Boolean = { x -> !x.contains("(") } + source.StaticField = listMerge(source.StaticField, this.keys.toList(), filter2) + } + } + } else { + ruleData.source?.run { + source.RuleObjReturn = listMerge( + source.RuleObjReturn, + if (Return is JsonArray) { + Json.decodeFromJsonElement>( + Return as JsonArray + ).toMutableList() + } else source.RuleObjReturn + ) + source.UseJSInterface = if (source.UseJSInterface) source.UseJSInterface else UseJSInterface + source.Param = mapMerge(source.Param, Param) + source.StaticField = listMerge(source.StaticField, StaticField) + source.ConstString = listMerge(source.ConstString, ConstString) + source.NewInstance = listMerge(source.NewInstance, NewInstance) + } + + } + } + + } + + private suspend fun loadRuleFromFile(ruleFile: String): JsonObject { + val curRulePath = "${getConfig().rulePath}/$ruleFile" + val jsonStr = Rules.loadConfigOrQuit(curRulePath) + return Json.parseToJsonElement(jsonStr).jsonObject + } + + private suspend fun parseSinkRuleObj(ruleObj: RuleObjBody, sink: Map?): Map { + var sink2 = sink?.toMap() ?: mapOf() + val secRules = loadRuleFromFile(ruleObj.ruleFile!!) + secRules.keys.forEach { ruleName -> + if (ruleObj.include.isNotEmpty() && !ruleObj.include.contains(ruleName)) { + return@forEach + } + val jsonObj = secRules[ruleName]!! + val ruleData: RuleData = Json.decodeFromJsonElement(jsonObj) + sink2 = mapMerge(sink2, ruleData.sink) + } + return sink2 + } + + private fun listMerge( + old: List, + other: List, + filter: (String) -> Boolean = { true } + ): List { + val new = ArrayList() + for (s in old) { + new.add(s) + } + for (s in other) { + if (!new.contains(s) && filter(s)) { + new.add(s) + } + } + return new + } + + private fun mapMerge( + old: Map, + other: Map?, + reduce: (V, V) -> V = { _, b -> b } + ): Map { + val new = HashMap() + for ((k, v) in old) { + new[k] = v + } + other?.forEach { (key, value) -> + new[key] = new[key]?.let { reduce(value, it) } ?: value + } + return new + } + + init { + entry = ruleData.entry + sinkRules = ruleData.sinkRuleObj + sourceRules = ruleData.sourceRuleObj + throughAPI = ruleData.throughAPI + } + + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/rules/IRule.kt b/src/main/kotlin/net/bytedance/security/app/rules/IRule.kt new file mode 100644 index 0000000..4bc4dea --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/rules/IRule.kt @@ -0,0 +1,47 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.rules + +import net.bytedance.security.app.RuleDescription + +/** + *Abstract interfaces to all rules, because the engine supports many modes, and modes can be extended as needed + */ +interface IRule { + /** + * SliceMode,apiMode etc + */ + val mode: String + + /** + * + * { + * "name": "DESEncryption", + * "category": "CryptoRisk", + * "wiki": "", + * "detail": "some description", + * "possibility": "4", + * "model": "low" + * } + */ + val desc: RuleDescription + + //name of rule + val name: String + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/rules/IRuleFactory.kt b/src/main/kotlin/net/bytedance/security/app/rules/IRuleFactory.kt new file mode 100644 index 0000000..17fb0bf --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/rules/IRuleFactory.kt @@ -0,0 +1,25 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.rules + +import net.bytedance.security.app.RuleData + + +interface IRuleFactory { + suspend fun create(name: String, ruleData: RuleData): IRule +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/rules/IRulesForContext.kt b/src/main/kotlin/net/bytedance/security/app/rules/IRulesForContext.kt new file mode 100644 index 0000000..ce89416 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/rules/IRulesForContext.kt @@ -0,0 +1,44 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.rules + +interface IRuleConstStringPattern { + /** + const string pattern in rule + */ + fun constStringPatterns(): Set + +} + +interface IRuleNewInstance { + /* +new instance in rules + */ + fun newInstances(): Set +} + +interface IRuleField { + /* + fields in rules + */ + fun fields(): Set +} + +interface IRulesForContext : IRuleConstStringPattern, IRuleNewInstance, IRuleField { + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/rules/RuleFactory.kt b/src/main/kotlin/net/bytedance/security/app/rules/RuleFactory.kt new file mode 100644 index 0000000..645344f --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/rules/RuleFactory.kt @@ -0,0 +1,39 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.rules + +import net.bytedance.security.app.RuleData + +open class RuleFactory : IRuleFactory { + override suspend fun create(name: String, ruleData: RuleData): IRule { + if (ruleData.ConstNumberMode == true) { + return ConstNumberModeRule(name, ruleData) + } else if (ruleData.ConstStringMode == true) { + return ConstStringModeRule(name, ruleData) + } else if (ruleData.SliceMode == true) { + val r = SliceModeRule(name, ruleData) + r.initIfNeeded() + return r + } else if (ruleData.DirectMode == true) { + val r = DirectModeRule(name, ruleData) + r.initIfNeeded() + return r + } + throw Exception("Unknown rule type $name") + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/rules/Rules.kt b/src/main/kotlin/net/bytedance/security/app/rules/Rules.kt new file mode 100644 index 0000000..24941a7 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/rules/Rules.kt @@ -0,0 +1,89 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.rules + +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.withContext +import kotlinx.serialization.json.jsonObject +import net.bytedance.security.app.Log +import net.bytedance.security.app.RuleData +import net.bytedance.security.app.util.Json +import java.io.IOException +import java.nio.file.Files +import java.nio.file.Paths + +class Rules(val rulePaths: List, val factory: IRuleFactory) : IRulesForContext { + val allRules: MutableList = ArrayList() + + suspend fun loadRules() { + rulePaths.forEach { + val jsonStr = loadConfigOrQuit(it) + val rules = Json.parseToJsonElement(jsonStr) + for ((ruleName, ruleBody) in rules.jsonObject) { + val ruleData: RuleData = Json.decodeFromJsonElement(ruleBody) + val rule = factory.create(ruleName, ruleData) + allRules.add(rule) + } + } + } + + override fun constStringPatterns(): Set { + val s = HashSet() + allRules.forEach { + if (it is IRuleConstStringPattern) + s.addAll(it.constStringPatterns()) + } + return s + } + + override fun newInstances(): Set { + val s = HashSet() + allRules.forEach { + if (it is IRuleNewInstance) + s.addAll(it.newInstances()) + } + return s + } + + override fun fields(): Set { + val s = HashSet() + allRules.forEach { + if (it is IRuleField) { + s.addAll(it.fields()) + } + } + return s + } + + companion object { + suspend fun loadConfigOrQuit(path: String): String { + Log.logInfo("Load config file $path") + val jsonStr = + withContext(Dispatchers.IO) { + try { + String(Files.readAllBytes(Paths.get(path))) + } catch (e: IOException) { + Log.logErr("read config file $path failed") + throw Exception("read config file $path failed") + } + } + + return jsonStr + } + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/rules/SliceModeRule.kt b/src/main/kotlin/net/bytedance/security/app/rules/SliceModeRule.kt new file mode 100644 index 0000000..5f78632 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/rules/SliceModeRule.kt @@ -0,0 +1,26 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.rules + +import net.bytedance.security.app.RuleData + +open class SliceModeRule(name: String, ruleData: RuleData) : DirectModeRule(name, ruleData) { + override val mode: String = "SliceMode" + val isSliceEnable: Boolean = true + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/rules/TaintFlowRule.kt b/src/main/kotlin/net/bytedance/security/app/rules/TaintFlowRule.kt new file mode 100644 index 0000000..a859913 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/rules/TaintFlowRule.kt @@ -0,0 +1,54 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.rules + +import kotlinx.serialization.json.JsonElement +import net.bytedance.security.app.* + + +abstract class TaintFlowRule(name: String, ruleData: RuleData) : AbstractRule(name, ruleData) { + + var sink: Map + val sanitize: Map>? + var source: SourceBody? + + val polymorphismBackTrace: Boolean + + var primTypeAsTaint: Boolean + val taintTweak: TaintTweakData? + + val traceDepth: Int + + init { + sink = ruleData.sink ?: emptyMap() + sanitize = ruleData.sanitize + source = ruleData.source + polymorphismBackTrace = ruleData.PolymorphismBackTrace == true + primTypeAsTaint = ruleData.PrimTypeAsTaint == true + taintTweak = ruleData.TaintTweak + traceDepth = ruleData.traceDepth!! + } + + fun isThisRuleNeedLog(): Boolean { + return getConfig().debugRule == this.name + } + + fun isThroughEnable(): Boolean { + return false + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/rules/TaintPosition.kt b/src/main/kotlin/net/bytedance/security/app/rules/TaintPosition.kt new file mode 100644 index 0000000..c1f83d0 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/rules/TaintPosition.kt @@ -0,0 +1,69 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.rules + +import net.bytedance.security.app.PLUtils +import net.bytedance.security.app.util.argIndex + +class TaintPosition(arg: String) { + + val position: Int + + init { + position = if (arg == PLUtils.THIS_FIELD || arg == BASE) { + This + } else if (arg == RETURN || arg == RET) { + Return + } else if (arg == BASE_DATA) { + ThisAllField + } else if (arg == MATCH_ALL) { + AllArgument + } else if (arg.startsWith("p")) { + arg.argIndex() + } else { + throw Exception("unknown taint point $arg") + } + } + + fun isConcreteArgument(): Boolean { + return position >= 0 + } + + companion object { + //all the argument + const val AllArgument = -1 + + //return of a method + const val Return = -2 + + //this pointer of a method + const val This = -3 + + //all field of this pointer + const val ThisAllField = -4 + + const val BASE = "@this" + const val BASE_DATA = "@this.data" + + // const val RET_DATA = "ret.data" + const val RET = "ret" + const val RETURN = "return" + const val MATCH_ALL = "p*" + } + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/sanitizer/ConstStringCheckSanitizer.kt b/src/main/kotlin/net/bytedance/security/app/sanitizer/ConstStringCheckSanitizer.kt new file mode 100644 index 0000000..55c5c9c --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/sanitizer/ConstStringCheckSanitizer.kt @@ -0,0 +1,35 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.sanitizer + +import net.bytedance.security.app.pointer.PLLocalPointer + +/** + * if these const strings are referenced, they are considered to satisfy the sanitizer condition + */ +class ConstStringCheckSanitizer(val constStrings: List) : + ISanitizer { + override fun matched(ctx: SanitizeContext): Boolean { + for (c in constStrings) { + if (ctx.ctx.pt.ptrIndexMap.contains(c.id)) { + return true + } + } + return false + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/sanitizer/ISanitizer.kt b/src/main/kotlin/net/bytedance/security/app/sanitizer/ISanitizer.kt new file mode 100644 index 0000000..6bcb39d --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/sanitizer/ISanitizer.kt @@ -0,0 +1,29 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.sanitizer + +import net.bytedance.security.app.pointer.PLPointer +import net.bytedance.security.app.taintflow.AnalyzeContext + +class SanitizeContext(val ctx: AnalyzeContext, val src: PLPointer?) +interface ISanitizer { + /** + * whether the ctx matches a sanitizer + */ + fun matched(ctx: SanitizeContext): Boolean +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/sanitizer/MethodCheckSanitizer.kt b/src/main/kotlin/net/bytedance/security/app/sanitizer/MethodCheckSanitizer.kt new file mode 100644 index 0000000..e35acc0 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/sanitizer/MethodCheckSanitizer.kt @@ -0,0 +1,34 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.sanitizer + +import soot.SootMethod + +/** + * If any of the methods are invoked, the sanitizer condition is considered satisfied + */ +class MethodCheckSanitizer(val methods: List) : ISanitizer { + override fun matched(ctx: SanitizeContext): Boolean { + for (m in methods) { + if (ctx.ctx.rm.contains(m)) { + return true + } + } + return false + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/sanitizer/MustNotPassSanitizer.kt b/src/main/kotlin/net/bytedance/security/app/sanitizer/MustNotPassSanitizer.kt new file mode 100644 index 0000000..cf4cec6 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/sanitizer/MustNotPassSanitizer.kt @@ -0,0 +1,27 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.sanitizer + +/** + * always false sanitizer + */ +class MustNotPassSanitizer : ISanitizer { + override fun matched(ctx: SanitizeContext): Boolean { + return false + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/sanitizer/MustPassSanitizer.kt b/src/main/kotlin/net/bytedance/security/app/sanitizer/MustPassSanitizer.kt new file mode 100644 index 0000000..cb3ba65 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/sanitizer/MustPassSanitizer.kt @@ -0,0 +1,28 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.sanitizer + +/** + * always pass sanitizer + */ +@Suppress("unused") +class MustPassSanitizer : ISanitizer { + override fun matched(ctx: SanitizeContext): Boolean { + return true + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/sanitizer/SanitizeOrRules.kt b/src/main/kotlin/net/bytedance/security/app/sanitizer/SanitizeOrRules.kt new file mode 100644 index 0000000..29f6da2 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/sanitizer/SanitizeOrRules.kt @@ -0,0 +1,33 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.sanitizer + +/** + * rules are or relations + */ +class SanitizeOrRules(val rules: List) : ISanitizer { + + override fun matched(ctx: SanitizeContext): Boolean { + for (r in rules) { + if (r.matched(ctx)) { + return true + } + } + return false + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/sanitizer/SanitizerAndRules.kt b/src/main/kotlin/net/bytedance/security/app/sanitizer/SanitizerAndRules.kt new file mode 100644 index 0000000..ffc16d3 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/sanitizer/SanitizerAndRules.kt @@ -0,0 +1,33 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.sanitizer + +/** + * and relations of rules + */ +class SanitizerAndRules(val rules: List) : ISanitizer { + + override fun matched(ctx: SanitizeContext): Boolean { + for (r in rules) { + if (!r.matched(ctx)) { + return false + } + } + return true + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/sanitizer/SanitizerFactory.kt b/src/main/kotlin/net/bytedance/security/app/sanitizer/SanitizerFactory.kt new file mode 100644 index 0000000..6a26647 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/sanitizer/SanitizerFactory.kt @@ -0,0 +1,155 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.sanitizer + +import kotlinx.serialization.json.JsonElement +import kotlinx.serialization.json.jsonObject +import net.bytedance.security.app.Log.logInfo +import net.bytedance.security.app.PLUtils +import net.bytedance.security.app.PreAnalyzeContext +import net.bytedance.security.app.SinkBody +import net.bytedance.security.app.pointer.PLLocalPointer +import net.bytedance.security.app.rules.TaintFlowRule +import net.bytedance.security.app.util.Json +import net.bytedance.security.app.util.isFieldSignature +import soot.RefType +import soot.Scene +import soot.jimple.internal.JAssignStmt +import soot.jimple.internal.JInstanceFieldRef +import soot.jimple.internal.JimpleLocal + +/** + * SanitizerFactory is used to create Sanitizer + * Because a rule can be used in many places, sanitizer is cached for each rule + */ +object SanitizerFactory { + private val cache = HashMap>() + + @Synchronized + fun createSanitizers( + rule: TaintFlowRule, + ctx: PreAnalyzeContext, + ): List { + if (cache.contains(rule.name)) { + return cache[rule.name]!! + } + val sanitizes = rule.sanitize ?: return listOf() + val result = ArrayList() + for ((_, sanitizeRules) in sanitizes) { + val andRules = ArrayList() + for ((key, body) in sanitizeRules) { + if (key == "ConstString") { + andRules.add(createConstStringSanitizer(body, ctx)) + } else if (key.isFieldSignature()) { + andRules.add(createFieldSanitizer(body, key, ctx, rule)) + } else { + val sinkBody: SinkBody = Json.decodeFromJsonElement(body) + val p = TaintCheckSanitizerParser(ctx, sinkBody, key, rule) + andRules.add(p.createMethodSanitizer()) + } + } + if (andRules.size == 1) { + result.add(andRules.first()) + } else if (andRules.size > 1) { + result.add(SanitizerAndRules(andRules)) + } else { + logInfo("no sanitizer for rule: ${rule.name}") + } + + } + cache[rule.name] = result + return result + } + + @Synchronized + fun clearCache() { + cache.clear() + } + + private fun createConstStringSanitizer(array: JsonElement, ctx: PreAnalyzeContext): ISanitizer { + val constStrings: List = Json.decodeFromJsonElement(array) + val pointers = ArrayList() + for (pattern in constStrings) { + val constCallMap = ctx.findConstStringPatternCallSite(pattern) + for (callsite in constCallMap) { + for (str in callsite.constString()) { + val ptr = PLLocalPointer( + callsite.method, + PLUtils.constStrSig(str), + RefType.v("java.lang.String") + ) + pointers.add(ptr) + } + } + } + return ConstStringCheckSanitizer(pointers) + } + + + /** + * create a sanitizer for a object field + * TaintCheck: + * - @this check this object of the field is tainted + * - @data check this field is tainted + "": { + "TaintCheck":["@this"] + } + */ + private fun createFieldSanitizer( + ruleObj: JsonElement, + fieldSig: String, + ctx: PreAnalyzeContext, + rule: TaintFlowRule + ): ISanitizer { + val field = Scene.v().grabField(fieldSig) + if (field == null || ruleObj.jsonObject.isEmpty() || field.isStatic) { + return MustNotPassSanitizer() + } + val sinkBody: SinkBody = Json.decodeFromJsonElement(ruleObj) + assert(sinkBody.TaintCheck != null) + var checkBase = false + var checkField = false + for (obj in sinkBody.TaintCheck!!) { + if (obj == PLUtils.THIS_FIELD) { + checkBase = true + } else if (obj == PLUtils.DATA_FIELD) { + checkField = true + } + } + if (!checkBase && !checkField) { + return MustNotPassSanitizer() + } + val callsites = ctx.findFieldCallSite(field) + val pointers = ArrayList() + for (callsite in callsites) { + val stmt = callsite.stmt + if (checkBase) { + val fieldRef = stmt.fieldRef as JInstanceFieldRef + val ptr = PLLocalPointer(callsite.method, (fieldRef.base as JimpleLocal).name, fieldRef.base.type) + pointers.add(ptr) + } + if (checkField) { + val ret = (stmt as? JAssignStmt)?.leftOp as? JimpleLocal ?: continue + val ptr = PLLocalPointer(callsite.method, ret.name, ret.type) + pointers.add(ptr) + } + } + return TaintCheckSanitizer(pointers.toSet(), emptySet(), emptyMap(), rule) + } + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/sanitizer/TaintCheckSanitizer.kt b/src/main/kotlin/net/bytedance/security/app/sanitizer/TaintCheckSanitizer.kt new file mode 100644 index 0000000..3cb0466 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/sanitizer/TaintCheckSanitizer.kt @@ -0,0 +1,153 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.sanitizer + +import net.bytedance.security.app.pointer.PLLocalPointer +import net.bytedance.security.app.rules.TaintFlowRule +import soot.SootMethod + +/** + * Taint tests are performed on the specified variables, + * requiring all `taints` are tainted and none of `notTaints` are tainted + * for example: +"": { +"TaintCheck":["@this"], +"p0": [1] +} + * @param constStrings check if any const strings flow to variables ( key of the map). + * Integer constants are converted to string + */ +class TaintCheckSanitizer( + val taints: Set, + val notTaints: Set, + val constStrings: Map>, + val rule: TaintFlowRule, +) : ISanitizer { + override fun matched(ctx: SanitizeContext): Boolean { + assert(checkAllPtrIsInOneMethod()) + val allTaints = ctx.ctx.collectPropagation(ctx.src!!, rule.primTypeAsTaint) + var taintedPass = taints.isEmpty() + for (p in taints) { + if (allTaints.contains(p)) { + taintedPass = true + break + } + } + if (!taintedPass) { + return false + } + var notTaintedPass = notTaints.isEmpty() + for (p in notTaints) { + if (allTaints.contains(p)) { + notTaintedPass = false + break + } + } + if (!notTaintedPass) { + return false + } + var constPass = constStrings.isEmpty() + found@ for ((dst, patterns) in constStrings) { + if (constPass) break + val allPatternTainted = ctx.ctx.collectReversePropagation(dst, rule.primTypeAsTaint) + for (ptr in allPatternTainted) { + if (ptr !is PLLocalPointer || !ptr.isConstStr) { + continue + } + val ptrValue = ptr.variableName + for (pattern in patterns) { + if (isSanitizeStrMatch(pattern, ptrValue)) { + constPass = true + break@found + } + } + } + } + if (!constPass) { + return false + } + return true + } + + fun checkAllPtrIsInOneMethod(): Boolean { + var method: SootMethod? = null + val pointers = taints.toMutableList() + notTaints.forEach { pointers.add(it) } + constStrings.forEach { pointers.add(it.key) } + pointers.forEach { + if (method == null) { + method = it.method + } else if (it.method != method) { + return false + } + } + return true + } + + companion object { + /** + The sanitizer rule has four forms for constants: + 1. Ordinary integers are converted to string for comparison. + 2. Ordinary strings are compared directly. + 3. for strings contains '*', a simple regular expression to match + 4. bits match mode + */ + fun isSanitizeStrMatch(pattern: String, target: String): Boolean { + val patternSub = pattern.replace("*", "") + return if (pattern.startsWith("*") && pattern.endsWith("*")) { + target.contains(patternSub) + } else if (pattern.startsWith("*")) { + target.endsWith(patternSub) + } else if (pattern.endsWith("*")) { + target.startsWith(patternSub) + } else { + if (pattern.endsWith(":&") || pattern.endsWith(":|")) { + isBitModeMatch(pattern, target) + } else { + target == patternSub + } + } + } + + /** + * bits match mode: + * 67108864:& means the target &67108864 !=0 + * 67108864:| means the target & 67108864==0 + * @param pattern like 67108864:& + * @param target must be a integer string + */ + private fun isBitModeMatch(pattern: String, target: String): Boolean { + try { + val l = target.toLong() + val isOr = pattern.endsWith(":|") + val isAnd = pattern.endsWith(":&") + val pl = pattern.slice(0 until pattern.length - 2).toLong() + if (isAnd) { + val s = l.and(pl) + return s != 0L + } + if (isOr) { + return l.and(pl) == 0L + } + throw Exception("unknown pattern $pattern") + } catch (ex: NumberFormatException) { + return false + } + } + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/sanitizer/TaintCheckSanitizerParser.kt b/src/main/kotlin/net/bytedance/security/app/sanitizer/TaintCheckSanitizerParser.kt new file mode 100644 index 0000000..20181b6 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/sanitizer/TaintCheckSanitizerParser.kt @@ -0,0 +1,240 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.sanitizer + +import kotlinx.serialization.json.JsonElement +import kotlinx.serialization.json.jsonPrimitive +import net.bytedance.security.app.MethodFinder +import net.bytedance.security.app.PreAnalyzeContext +import net.bytedance.security.app.SinkBody +import net.bytedance.security.app.pointer.PLLocalPointer +import net.bytedance.security.app.preprocess.CallSite +import net.bytedance.security.app.rules.TaintFlowRule +import net.bytedance.security.app.rules.TaintPosition +import soot.SootMethod +import soot.Value +import soot.jimple.* +import soot.jimple.internal.JimpleLocal + +class TaintCheckSanitizerParser( + private val ctx: PreAnalyzeContext, + private val sinkBody: SinkBody, + private val methodSig: String, + private val rule: TaintFlowRule, +) { + + /** + * + "": { + "TaintCheck": ["@this"], + "NotTaint":["p1"], + "p0":["..*"] + } + TaintCheck,NotTaint, and p0 are and relations, + multiple sanitizers are created here because they are for the same callsite. + */ + fun createMethodSanitizer(): ISanitizer { + val targetMethodSet = MethodFinder.checkAndParseMethodSig(methodSig) + val callsites = HashSet() + for (m in targetMethodSet) { + val callsites2 = ctx.findInvokeCallSite(m) + callsites.addAll(callsites2) + } + if (sinkBody.isEmpty()) { + return MethodCheckSanitizer(targetMethodSet.toList()) + } + val possibleMatches: MutableList = ArrayList() + for (callsite in callsites) { + val stmt = callsite.stmt + val callerMethod = callsite.method + val invokeExpr = stmt.invokeExpr + val taintPtrSet = HashSet() + val notTaintParamCheckSet = HashSet() + val taintArray = sinkBody.TaintCheck + val taintCheckPtrSet = calcSanitizePtrSet(taintArray, invokeExpr, callerMethod) + if (taintCheckPtrSet.isNotEmpty()) { + taintPtrSet.addAll(taintCheckPtrSet) + } + val cleanArray = sinkBody.NotTaint + val cleanCheckPtrSet = calcSanitizePtrSet(cleanArray, invokeExpr, callerMethod) + if (cleanCheckPtrSet.isNotEmpty()) { + notTaintParamCheckSet.addAll(cleanCheckPtrSet) + } + val constStrings: MutableMap> = HashMap() + + val isPossibleMatch = calcCheckConstStrToVariable(sinkBody, invokeExpr, callerMethod, constStrings) + if (!isPossibleMatch) { + continue + } + + val s = TaintCheckSanitizer(taintPtrSet, notTaintParamCheckSet, constStrings, rule) + assert(s.checkAllPtrIsInOneMethod()) + possibleMatches.add(s) + } + /// The relationship between different call sites is or, as long as one of them is satisfied, it is ok + return SanitizeOrRules(possibleMatches) + } + + /** + * find target variables for TaintCheck or NotTaint + * @param taintCheckArray like "TaintCheck": ["p*","@this"] + * or "NotTaint": ["p0", "p1"] + * + * @param invokeExpr for example : + * `virtualinvoke r1.("..")` + * @param callerMethod the method that contains the invokeExpr + * @return variables need to be checked + */ + private fun calcSanitizePtrSet( + taintCheckArray: List?, + invokeExpr: InvokeExpr, + callerMethod: SootMethod + ): Set { + val ptrSet: MutableSet = HashSet() + if (taintCheckArray == null) { + return ptrSet + } + for (taintParam in taintCheckArray) { + val taintPosition = TaintPosition(taintParam) + if (taintPosition.position == TaintPosition.This) { + if (invokeExpr is InstanceInvokeExpr) { + val base = invokeExpr.base as JimpleLocal + val ptr = PLLocalPointer( + callerMethod, + base.name, base.type + ) + ptrSet.add(ptr) + } + } else if (taintPosition.position == TaintPosition.AllArgument) { + for (arg in invokeExpr.args) { + if (arg is JimpleLocal) { + val ptr = PLLocalPointer( + callerMethod, + arg.name, arg.getType() + ) + ptrSet.add(ptr) + } + } + } else if (taintPosition.isConcreteArgument()) { + val i = taintPosition.position + if (i < invokeExpr.argCount) { + val arg = invokeExpr.getArg(i) + if (arg is JimpleLocal) { + val ptr = PLLocalPointer( + callerMethod, + arg.name, arg.getType() + ) + ptrSet.add(ptr) + } + } + } + } + return ptrSet + } + + /** + * @param sinkBody + * "": { + * "TaintCheck": ["@this"], + * "p0":["..*"] + * "p*":["..*"] + * } + * @param invokeExpr The expression that calls this function, for example: r1.contains(r3) + * @param callerMethod the method that contains the invokeExpr + * @param constStrings result if found,for example key is r3, and value is "..*" + * @return true if it is possible to match the sanitizer, false otherwise. + */ + private fun calcCheckConstStrToVariable( + sinkBody: SinkBody, + invokeExpr: InvokeExpr, + callerMethod: SootMethod, + constStrings: MutableMap> + ): Boolean { + if (sinkBody.pstar != null) { + for (arg in invokeExpr.args) { + if (!calcForOneArg(callerMethod, sinkBody.pstar, arg, constStrings)) { + return false + } + } + } else { + sinkBody.pmap?.forEach { + val index = it.key.slice(1 until it.key.length).toInt() + if (index < invokeExpr.argCount) { + val arg = invokeExpr.getArg(index) + if (!calcForOneArg(callerMethod, it.value, arg, constStrings)) { + return false + } + } + } + } + return true + } + + private fun calcForOneArg( + callerMethod: SootMethod, + constStrArray: List, + arg: Value, + constStrings: MutableMap> + ): Boolean { + val patterns = constStrArray.map { it.jsonPrimitive.content } + if (arg is Constant) { + var possible = false + for (pattern in patterns) { + if (TaintCheckSanitizer.isSanitizeStrMatch(pattern, arg.getStringValue())) { + possible = true + break + } + } + if (!possible) { + return false // this call site cannot match sanitizer + } + + } else if (arg is JimpleLocal) { + val ptr = PLLocalPointer(callerMethod, arg.name, arg.type) + constStrings[ptr] = patterns + } + return true + } +} + +/** + * + * If it is a string, return itself + * numeric constant, return its string + * null, return "null" + */ +fun Constant.getStringValue(): String { + when (this) { + is StringConstant -> { + return value + } + is IntConstant -> { + return value.toString() + } + is NullConstant -> { + return "null" + } + is LongConstant -> { + return value.toString() + } + else -> { +// throw IllegalArgumentException("unsupported constant type: ${this.javaClass.name}") + } + } + return this.toString() +} diff --git a/src/main/kotlin/net/bytedance/security/app/taintflow/AnalyzeContext.kt b/src/main/kotlin/net/bytedance/security/app/taintflow/AnalyzeContext.kt new file mode 100644 index 0000000..06566d1 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/taintflow/AnalyzeContext.kt @@ -0,0 +1,293 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.taintflow + +import net.bytedance.security.app.pointer.PLObject +import net.bytedance.security.app.pointer.PLPointer +import net.bytedance.security.app.pointer.PointerFactory +import net.bytedance.security.app.util.toFormatedString +import net.bytedance.security.app.util.toSortedMap +import soot.PrimType +import soot.SootMethod +import java.util.concurrent.ConcurrentHashMap + + +/** + * the result of pointer analyze. + */ +class AnalyzeContext(val pt: PointerFactory) { + // reachable methods + var rm: MutableSet = HashSet() + + // pointer to object set + var pointerToObjectSet: MutableMap> = HashMap() + + /** + specialinvoke $r0_1.(net.bytedance.security.app.bvaa.openConnection.A$1)>(null); + = $r0_1; + key is r0_1,value:[, + (net.bytedance.security.app.bvaa.openConnection.A$1)>->@this] + The key is the propagation source, and the value is the propagation destination + */ + val variableFlowGraph: MutableMap> = HashMap() + + /** + * As with the variableFlowGraph, key is the propagation destination and value is the propagation source + */ + private val reverseVariableFlowGraph: MutableMap> = HashMap() + + /** + * $r0 = ; + * key is ina,value is r0, pointer propagated from ina to r0 + */ + var pointerFlowGraph: MutableMap> = HashMap() + + fun dump(): String { + return """ + ptrToSet={${pointerToObjectSet.toSortedMap().toFormatedString()}}, + + + ptrFlowGraph={${pointerFlowGraph.toSortedMap().toFormatedString()}}, + + + taintPtrFlowGraph={${variableFlowGraph.toSortedMap().toFormatedString()}}, + """.trimIndent() + } + + fun baseInfo(): String { + return """ + rm=${rm.size}, + pointerToObjectSet={${pointerToObjectSet.size}}, + pointerFlowGraph={${pointerFlowGraph.size}}, + taintPtrFlowGraph={${variableFlowGraph.size}}, + objects=${pt.objIndexMap.size}, + pointers=${pt.ptrIndexMap.size} + """.trimIndent() + } + + private fun addTaintPFGEdge( + srcPtr: PLPointer, + dstPtr: PLPointer, + @Suppress("UNUSED_PARAMETER") isPrime: Boolean + ): Boolean { + addToPointFlowGraph(dstPtr, srcPtr, reverseVariableFlowGraph) + return addToPointFlowGraph(srcPtr, dstPtr, variableFlowGraph) + } + + /** + * If srcPtr does not point to dstPtr, it is added to pointerFlowGraph and then returns true, + * If srcPtr already points to dstPtr, return false + */ + private fun addPtrFlowEdge(srcPtr: PLPointer, dstPtr: PLPointer): Boolean { + return addToPointFlowGraph(srcPtr, dstPtr, pointerFlowGraph) + } + + /** + * If srcPtr does not point to dstPtr, it is added to pfg and then returns true, + * If srcPtr already points to dstPtr, return false + */ + private fun addToPointFlowGraph( + srcPtr: PLPointer, dstPtr: PLPointer, + pfg: MutableMap> + ): Boolean { + var dstSet = pfg[srcPtr] + if (dstSet == null) { + dstSet = HashSet() + pfg[srcPtr] = dstSet + } + if (!dstSet.contains(dstPtr)) { + dstSet.add(dstPtr) + return true + } + return false + } + + fun getPointToSet(ptr: PLPointer): Set? { + return pointerToObjectSet[ptr] + } + + fun isInPointToSet(ptr: PLPointer): Boolean { + return pointerToObjectSet.containsKey(ptr) + } + + fun addToPointToSet(ptr: PLPointer, obj: PLObject) { + val objSet = pointerToObjectSet.computeIfAbsent(ptr) { HashSet() } + objSet.add(obj) + } + + private fun addToPointToSet(ptr: PLPointer, objs: Set) { + val objSet = pointerToObjectSet.computeIfAbsent(ptr) { HashSet() } + objSet.addAll(objs) + } + + private fun propagate(srcPtr: PLPointer, obj: PLObject) { + val dstPointers = pointerFlowGraph[srcPtr] + if (dstPointers != null) { + for (dstPtr in dstPointers) { + var dstObjects = pointerToObjectSet[dstPtr] + if (dstObjects == null) { + dstObjects = HashSet() + pointerToObjectSet[dstPtr] = dstObjects + dstObjects.add(obj) + } else { + if (!dstObjects.contains(obj)) { + dstObjects.add(obj) + propagate(dstPtr, obj) + } + } + } + } + } + + + private fun propagate(srcPtr: PLPointer, objs: Set) { + val dstPtrs = pointerFlowGraph[srcPtr] ?: return + for (dstPtr in dstPtrs) { + var dstObjs = pointerToObjectSet[dstPtr] + if (dstObjs == null) { + dstObjs = HashSet() + pointerToObjectSet[dstPtr] = dstObjs + dstObjs.addAll(objs) + propagate(dstPtr, objs) + } else { + for (obj in objs) { + if (!dstObjs.contains(obj)) { + dstObjs.addAll(objs) + propagate(dstPtr, objs) + break + } + } + } + } + + } + + private fun propagateObjs(srcPtr: PLPointer, dstPtr: PLPointer) { + val objs = pointerToObjectSet[srcPtr] + if (objs == null || objs.isEmpty()) { + return + } + addToPointToSet(dstPtr, objs) + + propagate(dstPtr, objs) + } + + + private fun propagateObj(srcPtr: PLPointer, obj: PLObject) { + propagate(srcPtr, obj) + } + + /** + * Add an edge to the pointerFlowGraph and propagate it if isPointerNeedPropagate is true. + * add an edge to the variableFlowGraph + * @param isPointerNeedPropagate indicating that the pointer relationship needs to be propagated + */ + + fun addPtrEdge(srcPtr: PLPointer, dstPtr: PLPointer, isPointerNeedPropagate: Boolean = true) { + addVariableFlowEdge(srcPtr, dstPtr) + if (addPtrFlowEdge(srcPtr, dstPtr)) { + if (isPointerNeedPropagate) { + propagateObjs(srcPtr, dstPtr) + } + } + } + + + fun addVariableFlowEdge(srcPtr: PLPointer, dstPtr: PLPointer, isPrime: Boolean) { + addTaintPFGEdge(srcPtr, dstPtr, isPrime) + } + + fun addVariableFlowEdge(srcPtr: PLPointer, dstPtr: PLPointer) { + if (isPrimePtr(srcPtr) || isPrimePtr(dstPtr)) { + addVariableFlowEdge(srcPtr, dstPtr, true) + } else { + addVariableFlowEdge(srcPtr, dstPtr, false) + } + } + + + fun addObjToPTS(srcPtr: PLPointer, obj: PLObject) { + addToPointToSet(srcPtr, obj) + propagateObj(srcPtr, obj) + } + + + private val propagationCache = ConcurrentHashMap>() + + + fun collectPropagation( + src: PLPointer, + isIncludePrimeTaint: Boolean = false + ): Set { + val key = "${src.signature()}:$isIncludePrimeTaint:0" + propagationCache[key]?.let { + return it + } + val s = collectPropagationInternal(src, this.variableFlowGraph, isIncludePrimeTaint) + propagationCache[key] = s + return s + } + + fun collectReversePropagation( + dst: PLPointer, + isIncludePrimeTaint: Boolean = false + ): Set { + val key = "${dst.signature()}:$isIncludePrimeTaint}:1" + propagationCache[key]?.let { + return it + } + val s = collectPropagationInternal(dst, this.reverseVariableFlowGraph, isIncludePrimeTaint) + propagationCache[key] = s + return s + } + + private fun collectPropagationInternal( + src: PLPointer, + graph: Map>, + isIncludePrimeTaint: Boolean = false + ): Set { + val result = HashSet() + val next = ArrayList() + next.add(src) + while (next.isNotEmpty()) { + val p = next.removeLast() + if (result.contains(p)) { + continue + } + result.add(p) + graph[p]?.let { candidates -> + for (n in candidates) { + if (result.contains(n)) { + continue + } + if (!isIncludePrimeTaint && (isPrimePtr(p) || isPrimePtr(n))) { + continue + } + next.add(n) + } + } + } + return result + } + + companion object { + fun isPrimePtr(ptr: PLPointer): Boolean { + return ptr.ptrType is PrimType + } + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/taintflow/DefaultMethodAnalyzeMode.kt b/src/main/kotlin/net/bytedance/security/app/taintflow/DefaultMethodAnalyzeMode.kt new file mode 100644 index 0000000..bedb381 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/taintflow/DefaultMethodAnalyzeMode.kt @@ -0,0 +1,36 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.taintflow + +import net.bytedance.security.app.engineconfig.EngineConfig +import net.bytedance.security.app.engineconfig.isLibraryClass +import soot.SootMethod + +object DefaultMethodAnalyzeMode : IMethodAnalyzeMode { + override fun methodMode(method: SootMethod): MethodAnalyzeMode { + if (EngineConfig.IgnoreListConfig.isInIgnoreList(method.declaringClass.name, method.name, method.signature)) { + return MethodAnalyzeMode.Skip + } else if (isLibraryClass(method.declaringClass.name)) { + return MethodAnalyzeMode.Obscure + } else if (!method.hasActiveBody()) { + return MethodAnalyzeMode.Obscure + } + return MethodAnalyzeMode.Analyze + } + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/taintflow/DefaultPointerPropagationRule.kt b/src/main/kotlin/net/bytedance/security/app/taintflow/DefaultPointerPropagationRule.kt new file mode 100644 index 0000000..b3aa106 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/taintflow/DefaultPointerPropagationRule.kt @@ -0,0 +1,44 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.taintflow + +import net.bytedance.security.app.engineconfig.PointerPropagationConfig +import soot.SootMethod + +/** + * rule from the configuration file + */ +class DefaultPointerPropagationRule(cfg: PointerPropagationConfig) : IPointerFlowRule { + private val methodRule: Map> + + init { + this.methodRule = + cfg.methodNameRule.mapValues { it.value.toFlowItem() } + cfg.methodSigRule.mapValues { it.value.toFlowItem() } + } + + override fun flow(calleeMethod: SootMethod): List? { + methodRule[calleeMethod.signature]?.let { + return it + } + methodRule[calleeMethod.name]?.let { + return it + } + return null + } + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/taintflow/DefaultVariableFlowRule.kt b/src/main/kotlin/net/bytedance/security/app/taintflow/DefaultVariableFlowRule.kt new file mode 100644 index 0000000..1c7be78 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/taintflow/DefaultVariableFlowRule.kt @@ -0,0 +1,52 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.taintflow + +import net.bytedance.security.app.engineconfig.VariableFlowConfig +import soot.SootMethod + +/** + * rule from the configuration file + */ +class DefaultVariableFlowRule(cfg: VariableFlowConfig) : + IVariableFlowRule { + private val instantRule: List + private val instantSelfRule: List + private val methodRule: Map> + + init { + instantRule = cfg.instantDefaultRule.toFlowItem() + instantSelfRule = cfg.instantSelfDefaultRule.toFlowItem() + this.methodRule = cfg.methodNameRule.mapValues { it.value.toFlowItem() } + + cfg.methodSigRule.mapValues { it.value.toFlowItem() } + } + + + override fun flow(callerMethod: SootMethod, calleeMethod: SootMethod): List { + methodRule[calleeMethod.signature]?.let { + return it + } + methodRule[calleeMethod.name]?.let { + return it + } + if (callerMethod.declaringClass == calleeMethod.declaringClass && !calleeMethod.isStatic) { + return instantSelfRule + } + return instantRule + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/taintflow/IMethodAnalyzeMode.kt b/src/main/kotlin/net/bytedance/security/app/taintflow/IMethodAnalyzeMode.kt new file mode 100644 index 0000000..95aacfb --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/taintflow/IMethodAnalyzeMode.kt @@ -0,0 +1,24 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.taintflow + +import soot.SootMethod + +interface IMethodAnalyzeMode { + fun methodMode(method: SootMethod): MethodAnalyzeMode +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/taintflow/IVariableFlowRule.kt b/src/main/kotlin/net/bytedance/security/app/taintflow/IVariableFlowRule.kt new file mode 100644 index 0000000..163dbc1 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/taintflow/IVariableFlowRule.kt @@ -0,0 +1,52 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.taintflow + +import net.bytedance.security.app.engineconfig.Propagation +import net.bytedance.security.app.rules.TaintPosition +import soot.SootMethod + +/** + * taint flow from `from` to `to` + */ +data class FlowItem(val from: TaintPosition, val to: TaintPosition) + +fun List.toFlowItem(): List { + return this.map { FlowItem(TaintPosition(it.from), TaintPosition(it.to)) }.toList() +} + +/** + * taint flow by user defined rule + */ +interface IVariableFlowRule { + /** + * how to process variable flow when callerMethod calls a calleeMethod + */ + fun flow(callerMethod: SootMethod, calleeMethod: SootMethod): List +} + +/** + * pointer propagation by user defined rule + */ +interface IPointerFlowRule { + /** + * how to process pointer flow when callerMethod calls a calleeMethod + */ + fun flow(calleeMethod: SootMethod): List? +} + diff --git a/src/main/kotlin/net/bytedance/security/app/taintflow/MethodAnalyzeMode.kt b/src/main/kotlin/net/bytedance/security/app/taintflow/MethodAnalyzeMode.kt new file mode 100644 index 0000000..2df6ee4 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/taintflow/MethodAnalyzeMode.kt @@ -0,0 +1,35 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.taintflow + +enum class MethodAnalyzeMode { + /** + * This method call is treated as if it does not exist + */ + Skip, + + /** + * The pointer relations and variable flow relations are calculated by user defined rules. + */ + Obscure, + + /** + * This method needs to be analyzed instruction by instruction + */ + Analyze +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/taintflow/ObscureRuleHandler.kt b/src/main/kotlin/net/bytedance/security/app/taintflow/ObscureRuleHandler.kt new file mode 100644 index 0000000..3aedca9 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/taintflow/ObscureRuleHandler.kt @@ -0,0 +1,393 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +@file:Suppress("unused") + +package net.bytedance.security.app.taintflow + +import net.bytedance.security.app.PLUtils +import net.bytedance.security.app.getConfig +import net.bytedance.security.app.pointer.PLPointer +import net.bytedance.security.app.pointer.PointerFactory +import net.bytedance.security.app.rules.TaintPosition +import soot.UnknownType +import soot.jimple.Stmt + +class ObscureRuleHandler( + val ctx: AnalyzeContext, + private val pt: PointerFactory, +) { + /** + * add relations between pointer by rules in [ruleList] + * @param stmt the call stmt,for example,r=o.f(a1,a2,a3) + * @param basePointer o in r=o.f(a1,a2,a3) + * @param ruleList the rules to add relations + * @param baseDataPointers o.@data + * @param receivePointer r in r=o.f(a1,a2,a3) + * @param argPointers a1,a2,a3 in r=o.f(a1,a2,a3) + * @param isPointerRule whether the rule is pointer rule or variable flow rule + */ + fun addEdgeByRule( + stmt: Stmt, + ruleList: List, + basePointer: PLPointer?, + baseDataPointers: Set?, + receivePointer: PLPointer?, +// recvDataPtrs: Set, + argPointers: List, + isPointerRule: Boolean + ) { + for (entry in ruleList) { + val (from, to) = entry + when (from.position) { + TaintPosition.This -> when (to.position) { + TaintPosition.Return -> baseToReturnEdge(isPointerRule, stmt, basePointer, receivePointer) +// Wrapper.RET_DATA -> baseToReturnDataEdge(isPointerRule, stmt, basePtr, recvDataPtrs) + TaintPosition.AllArgument -> baseToAllArgEdge(isPointerRule, stmt, basePointer, argPointers) + else -> { + val index = to.position + if (index >= argPointers.size) { + break + } + baseToArgEdge(isPointerRule, stmt, basePointer, argPointers[index]) + } + } + TaintPosition.ThisAllField -> when (to.position) { + TaintPosition.Return -> baseDataToReturnEdge(isPointerRule, stmt, baseDataPointers, receivePointer) +// Wrapper.RET_DATA -> baseDataToReturnDataEdge(isPointerRule, stmt, baseDataPtrs, recvDataPtrs) + TaintPosition.AllArgument -> baseDataToArgEdge(isPointerRule, stmt, baseDataPointers, argPointers) + else -> { + val index = to.position + if (index >= argPointers.size) { + break + } + baseDataToArgEdge(isPointerRule, stmt, baseDataPointers, argPointers[index]) + } + } + TaintPosition.AllArgument -> when (to.position) { + TaintPosition.This -> argToBaseEdge(isPointerRule, stmt, argPointers, basePointer) + TaintPosition.ThisAllField -> argToBaseDataEdge(isPointerRule, stmt, argPointers, baseDataPointers) + TaintPosition.Return -> argToRetEdge(isPointerRule, stmt, argPointers, receivePointer) +// Wrapper.RET_DATA -> argToRetDataEdge(isPointerRule, stmt, argPtrs, recvDataPtrs) + } + else -> { + val fromIndex = from.position + if (fromIndex >= argPointers.size) { + break + } + when (to.position) { + TaintPosition.This -> argToBaseEdge(isPointerRule, stmt, argPointers[fromIndex], basePointer) + TaintPosition.ThisAllField -> argToBaseDataEdge( + isPointerRule, + stmt, + argPointers[fromIndex], + baseDataPointers + ) + TaintPosition.Return -> argToRetEdge( + isPointerRule, + stmt, + argPointers[fromIndex], + receivePointer + ) +// Wrapper.RET_DATA -> argToRetDataEdge(isPointerRule, stmt, argPtrs[i_index], recvDataPtrs) + else -> { + // p1 -> p2 + val toIndex = to.position + assert(fromIndex != toIndex) + if (toIndex >= argPointers.size) { + break + } + argToArgEdge(isPointerRule, stmt, argPointers[fromIndex], argPointers[toIndex]) + } + } + } + } + } + } + + + /* + * base.func(arg1, arg2) + * arg1 -> base.'@data' + * arg2 -> base.'@data' + * */ + private fun argToBaseDataEdge( + isPointerRule: Boolean, + stmt: Stmt, + argPtrs: List, + baseDataPtrs: Set? + ) { + if (baseDataPtrs == null) { + return + } + for (baseDataPtr in baseDataPtrs) { + for (argPtr in argPtrs) { + addEdge(isPointerRule, argPtr, baseDataPtr, stmt) + } + } + } + + private fun argToBaseDataEdge( + isPointerRule: Boolean, + stmt: Stmt, + argPtr: PLPointer, + baseDataPtrs: Set? + ) { + if (baseDataPtrs == null) { + return + } + for (baseDataPtr in baseDataPtrs) { + addEdge(isPointerRule, argPtr, baseDataPtr, stmt) + } + } + + /* + * base.func(arg1, arg2) + * arg1 -> base + * arg2 -> base + * */ + private fun argToBaseEdge(isPointerRule: Boolean, stmt: Stmt, argPtrs: List, basePtr: PLPointer?) { + if (basePtr == null) { + return + } + for (argPtr in argPtrs) { + addEdge(isPointerRule, argPtr, basePtr, stmt) + } + } + + private fun argToBaseEdge(isPointerRule: Boolean, stmt: Stmt, argPtr: PLPointer, basePtr: PLPointer?) { + if (basePtr == null) { + return + } + addEdge(isPointerRule, argPtr, basePtr, stmt) + } + + private fun argToRetEdge(isPointerRule: Boolean, stmt: Stmt, argPtrs: List, recvPtr: PLPointer?) { + for (argPtr in argPtrs) { + addEdge(isPointerRule, argPtr, recvPtr, stmt) + } + } + + private fun argToRetEdge(isPointerRule: Boolean, stmt: Stmt, argPtr: PLPointer, recvPtr: PLPointer?) { + addEdge(isPointerRule, argPtr, recvPtr, stmt) + } + + private fun argToArgEdge(isPointerRule: Boolean, stmt: Stmt, argPtr1: PLPointer, argPtr2: PLPointer) { + addEdge(isPointerRule, argPtr1, argPtr2, stmt) + } + + private fun argDataToArgEdge(isPointerRule: Boolean, stmt: Stmt, argPtr1: PLPointer, argPtr2: PLPointer?) { + if (!ctx.isInPointToSet(argPtr1)) { + return + } + for (obj1 in ctx.getPointToSet(argPtr1)!!) { + val argObjPtr1: PLPointer = pt.allocObjectField(obj1, PLUtils.DATA_FIELD, UnknownType.v()) + addEdge(isPointerRule, argObjPtr1, argPtr2, stmt) + } + } + + private fun argToArgDataEdge(isPointerRule: Boolean, stmt: Stmt, argPtr1: PLPointer, argPtr2: PLPointer) { + if (!ctx.isInPointToSet(argPtr2)) { + return + } + for (obj2 in ctx.getPointToSet(argPtr2)!!) { + val argObjPtr2: PLPointer = pt.allocObjectField(obj2, PLUtils.DATA_FIELD, UnknownType.v()) + addEdge(isPointerRule, argPtr1, argObjPtr2, stmt) + } + } + + private fun argDataToArgDataEdge(isPointerRule: Boolean, stmt: Stmt, argPtr1: PLPointer, argPtr2: PLPointer) { + if (!ctx.isInPointToSet(argPtr1) || !ctx.isInPointToSet(argPtr2)) { + return + } + for (obj1 in ctx.getPointToSet(argPtr1)!!) { + val argObjPtr1: PLPointer = pt.allocObjectField(obj1, PLUtils.DATA_FIELD, UnknownType.v()) + for (obj2 in ctx.getPointToSet(argPtr2)!!) { + val argObjPtr2: PLPointer = pt.allocObjectField(obj2, PLUtils.DATA_FIELD, UnknownType.v()) + addEdge(isPointerRule, argObjPtr1, argObjPtr2, stmt) + } + } + } + + private fun argToRetDataEdge( + isPointerRule: Boolean, + stmt: Stmt, + argPtrs: List, + recvDataPtrs: Set + ) { + for (argPtr in argPtrs) { + for (recvDataPtr in recvDataPtrs) { + addEdge(isPointerRule, argPtr, recvDataPtr, stmt) + } + } + } + + private fun argToRetDataEdge(isPointerRule: Boolean, stmt: Stmt, argPtr: PLPointer, recvDataPtrs: Set) { + for (recvDataPtr in recvDataPtrs) { + addEdge(isPointerRule, argPtr, recvDataPtr, stmt) + } + } + + /* + * ret = base.func(arg1, arg2) + * base.'@data' -> ret + * */ + private fun baseDataToReturnEdge( + isPointerRule: Boolean, + stmt: Stmt, + baseDataPtrs: Set?, + recvPtr: PLPointer? + ) { + if (baseDataPtrs == null) { + return + } + if (recvPtr == null) { + return + } + for (baseDataPtr in baseDataPtrs) { + addEdge(isPointerRule, baseDataPtr, recvPtr, stmt) + } + } + + private fun baseDataToReturnDataEdge( + isPointerRule: Boolean, + stmt: Stmt, + baseDataPtrs: Set?, + recvDataPtrs: Set + ) { + if (baseDataPtrs == null) { + return + } + for (baseDataPtr in baseDataPtrs) { + for (recvDataPtr in recvDataPtrs) { + addEdge(isPointerRule, baseDataPtr, recvDataPtr, stmt) + } + } + } + + /* + * ret = base.func(arg1, arg2) + * base.'@data' -> arg1 + * base.'@data' -> arg2 + * */ + private fun baseDataToArgEdge( + isPointerRule: Boolean, + stmt: Stmt, + baseDataPtrs: Set?, + argPtrs: List + ) { + if (baseDataPtrs == null) { + return + } + for (baseDataPtr in baseDataPtrs) { + for (argPtr in argPtrs) { + addEdge(isPointerRule, baseDataPtr, argPtr, stmt) + } + } + } + + private fun baseDataToArgEdge( + isPointerRule: Boolean, + stmt: Stmt, + baseDataPtrs: Set?, + argPtr: PLPointer + ) { + if (baseDataPtrs == null) { + return + } + for (baseDataPtr in baseDataPtrs) { + addEdge(isPointerRule, baseDataPtr, argPtr, stmt) + } + } + + /* + * ret = base.func(arg1, arg2) + * base -> arg1 + * base -> arg2 + * */ + private fun baseToAllArgEdge(isPointerRule: Boolean, stmt: Stmt, basePtr: PLPointer?, argPtrs: List) { + if (basePtr == null) { + return + } + for (argPtr in argPtrs) { + addEdge(isPointerRule, basePtr, argPtr, stmt) + } + } + + /** + * ret = base.func(arg1, arg2) + * base -> arg1 + */ + private fun baseToArgEdge(isPointerRule: Boolean, stmt: Stmt, basePtr: PLPointer?, argPtr: PLPointer) { + if (basePtr == null) { + return + } + addEdge(isPointerRule, basePtr, argPtr, stmt) + } + + /* + * ret = base.func(arg1, arg2) + * base -> ret + * */ + private fun baseToReturnEdge(isPointerRule: Boolean, stmt: Stmt, basePtr: PLPointer?, recvPtr: PLPointer?) { + if (recvPtr == null) { + return + } + if (basePtr == null) { + return + } + addEdge(isPointerRule, basePtr, recvPtr, stmt) + } + + /** + * ret = base.func(arg1, arg2) + * base-> ret.data + * for example: + * String s; + * char[] chars=s.toCharArray(); + */ + private fun baseToReturnDataEdge( + isPointerRule: Boolean, + stmt: Stmt, + basePtr: PLPointer?, + recvDataPtrs: Set + ) { + if (basePtr == null) { + return + } + for (recvDataPtr in recvDataPtrs) { + addEdge(isPointerRule, basePtr, recvDataPtr, stmt) + } + } + + private fun addEdge( + isPointerRule: Boolean, + srcPtr: PLPointer, + dstPtr: PLPointer?, + @Suppress("UNUSED_PARAMETER") stmt: Stmt + ) { + if (dstPtr == null) { + return + } + if (isPointerRule) { + ctx.addPtrEdge(srcPtr, dstPtr, !getConfig().skipPointerPropagationForLibraryMethod) + } else { + ctx.addVariableFlowEdge(srcPtr, dstPtr) + } + } + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/taintflow/PruneMethodAnalyzeMode.kt b/src/main/kotlin/net/bytedance/security/app/taintflow/PruneMethodAnalyzeMode.kt new file mode 100644 index 0000000..9243c2e --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/taintflow/PruneMethodAnalyzeMode.kt @@ -0,0 +1,93 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.taintflow + +import net.bytedance.security.app.PreAnalyzeContext +import net.bytedance.security.app.pointer.PLLocalPointer +import soot.SootMethod + +/** + * skip methods has no relation with sinks and sources + */ +class PruneMethodAnalyzeMode( + ctx: PreAnalyzeContext, + pointers: List, + depth: Int, + private val defaultMethodAnalyzeMode: IMethodAnalyzeMode +) : + IMethodAnalyzeMode { + private val includedMethods: Set + + init { + this.includedMethods = buildIncludedMethods(ctx, pointers, depth) + } + + + override fun methodMode(method: SootMethod): MethodAnalyzeMode { + if (!includedMethods.contains(method)) { + return MethodAnalyzeMode.Obscure + } + return defaultMethodAnalyzeMode.methodMode(method) + } + + companion object { + private fun buildIncludedMethods( + ctx: PreAnalyzeContext, + pointers: List, + depth: Int + ): Set { + val result = HashSet() + pointers.forEach { queryCallers(ctx, it.method, depth, result) } + + return result + } + + private fun queryCallers( + ctx: PreAnalyzeContext, + callee: SootMethod, + depth: Int, + result: MutableSet + ) { + result.add(callee) + if (depth == 0) { + return + } + val nextCallers = ctx.callGraph.heirReverseCallGraph[callee] ?: return + for (nextCaller in nextCallers) { + if (result.contains(nextCaller)) { + return + } + queryCallers(ctx, nextCaller, depth - 1, result) + } + } + + //make sure all the analyzers have the same rule + fun fromTaintAnalyzers( + analyzers: List, + depth: Int, + ctx: PreAnalyzeContext + ): PruneMethodAnalyzeMode { + val pointers = HashSet() + analyzers.forEach { + pointers.addAll(it.sinkPtrSet) + pointers.addAll(it.sourcePtrSet) + } + return PruneMethodAnalyzeMode(ctx, pointers.toList(), depth, DefaultMethodAnalyzeMode) + } + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/taintflow/StmtTransfer.kt b/src/main/kotlin/net/bytedance/security/app/taintflow/StmtTransfer.kt new file mode 100644 index 0000000..337d23b --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/taintflow/StmtTransfer.kt @@ -0,0 +1,370 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.taintflow + +import net.bytedance.security.app.Log +import net.bytedance.security.app.PLUtils +import net.bytedance.security.app.engineconfig.isLibraryClass +import net.bytedance.security.app.pointer.PLLocalPointer +import net.bytedance.security.app.pointer.PLPointer +import net.bytedance.security.app.pointer.PointerFactory +import soot.SootMethod +import soot.UnknownType +import soot.Value +import soot.jimple.* +import soot.jimple.internal.* + + +/** + * handles statements other than function calls + */ +class StmtTransfer(val ctx: AnalyzeContext, private val pt: PointerFactory, private val tsp: TwoStagePointerAnalyze) { + + /* + * l2 := @parameter1: int + * l3 := @this: int + * + * @parameter1 -> l2 + * @this -> l3 + * + */ + fun identityStmt(identityStmt: JIdentityStmt, method: SootMethod) { + val leftOp = identityStmt.leftOp as JimpleLocal + val leftPtr: PLPointer = pt.allocLocal(method, leftOp.name, leftOp.type) + val rightOp = identityStmt.rightOp + val rightPtr = if (rightOp is ThisRef) { + // this := @this: Foo; + pt.allocLocal(method, PLUtils.THIS_FIELD, rightOp.getType()) + } else if (rightOp is ParameterRef) { + // a := @parameter0: java.lang.String; + pt.allocLocal(method, PLUtils.PARAM + rightOp.index, rightOp.type) + } else { // JCaughtExceptionRef + val jCaughtExceptionRef = rightOp as JCaughtExceptionRef + pt.allocLocal(method, jCaughtExceptionRef.toString(), jCaughtExceptionRef.type) + } + ctx.addPtrEdge(rightPtr, leftPtr) + } + + // left = op1 bin op2 + fun binaryOp(leftExpr: JimpleLocal, rightExpr: AbstractBinopExpr, method: SootMethod) { + val leftPtr: PLPointer = pt.allocLocal(method, leftExpr.name, leftExpr.type) + val op1 = rightExpr.op1 + val op2 = rightExpr.op2 + if (op1 !is NumericConstant) { + val local = op1 as JimpleLocal + val rightPtr: PLPointer = pt.allocLocal(method, local.name, local.type) + ctx.addVariableFlowEdge(rightPtr, leftPtr, true) + + } + if (op2 !is NumericConstant) { + val local = op2 as JimpleLocal + val rightPtr: PLPointer = pt.allocLocal(method, local.name, local.type) + ctx.addVariableFlowEdge(rightPtr, leftPtr, true) + + } + if (op1 is Constant) { + tsp.addConstValue(leftPtr, op1, method) + } + if (op2 is Constant) { + tsp.addConstValue(leftPtr, op2, method) + } + + } + + // $i1 = lengthof $r1 + // $i1 = neg $i0 + fun unaryOp(leftExpr: JimpleLocal, rightExpr: UnopExpr, method: SootMethod) { + val leftPtr: PLPointer = pt.allocLocal(method, leftExpr.name, leftExpr.type) + if (rightExpr.op is JimpleLocal) { + val op = rightExpr.op as JimpleLocal + val rightPtr: PLPointer = pt.allocLocal(method, op.name, op.type) + ctx.addVariableFlowEdge(rightPtr, leftPtr, true) + + } else { + Log.logErr("unaryOp $rightExpr") + } + } + + /* + * a = (B)b + * */ + fun castExpr(leftExpr: JimpleLocal, rightExpr: JCastExpr, method: SootMethod) { + val leftPtr = pt.allocLocal(method, leftExpr.name, leftExpr.type) + val rightOp = rightExpr.op + val castType = rightExpr.castType + if (rightOp is Constant) { // NumericConstant,StringConstant,NullConstant,ClassConstant + tsp.addConstValue(leftPtr, rightOp, method) + } else { // JimpleLocal + val rightPtr: PLPointer = + pt.allocLocal(method, (rightOp as JimpleLocal).name, rightOp.getType()) + ctx.addPtrEdge(rightPtr, leftPtr) + + tsp.makeNewObj(castType, rightOp, leftPtr) + } + } + + /* + left = base[2] + */ + fun loadArray(leftExpr: JimpleLocal, rightExpr: JArrayRef, method: SootMethod) { + val leftType = leftExpr.type + val leftPtr: PLPointer = pt.allocLocal(method, leftExpr.name, leftType) + val baseName = rightExpr.base.toString() + val baseType = rightExpr.base.type + val indexName = rightExpr.index.toString() + val indexType = rightExpr.index.type + val basePtr = pt.allocLocal(method, baseName, baseType) + val indexPtr = pt.allocLocal(method, indexName, indexType) + + var baseObjs = ctx.getPointToSet(basePtr) + if (baseObjs == null) { + baseObjs = tsp.makeNewObj(rightExpr.base.type, rightExpr, basePtr) + } + for (obj in baseObjs) { + val baseObjPtr: PLPointer = pt.allocObjectField(obj, PLUtils.DATA_FIELD, UnknownType.v()) + ctx.addPtrEdge(baseObjPtr, leftPtr) + } + ctx.addPtrEdge(basePtr, leftPtr) + ctx.addVariableFlowEdge(indexPtr, leftPtr, true) + } + + + /** + * base.field=right + */ + fun storeInstanceLocal( + leftOp: JInstanceFieldRef, + rightPtr: PLPointer, + method: SootMethod + ): PLLocalPointer { + val leftBaseName = leftOp.base.toString() + val leftBaseType = leftOp.base.type + val sootField = leftOp.field + val leftFieldName = sootField.name + + val leftBasePtr = pt.allocLocal(method, leftBaseName, leftBaseType) + var objs = ctx.getPointToSet(leftBasePtr) + if (objs == null) { + objs = tsp.makeNewObj(leftBaseType, leftOp, leftBasePtr) + } + val dataPointers = HashSet() + for (obj in HashSet(objs)) { + val leftFieldPtr = + pt.allocObjectField(obj, leftFieldName, sootField.type, sootField) + // a.b = c + // c -> o.b flow to + ctx.addPtrEdge(rightPtr, leftFieldPtr) + + /* + a.b=c , + the field that a.@data points to also needs to be merged, otherwise the taint will be lost. + */ + val dataPtr = pt.allocObjectField(obj, PLUtils.DATA_FIELD, UnknownType.v()) + if (ctx.pointerFlowGraph.containsKey(dataPtr)) { + dataPointers.add(dataPtr) +// ctx.addPtrEdge(rightPtr, leftBasePtr) java.util.ConcurrentModificationException + } + } + if (dataPointers.size > 0) { + ctx.addPtrEdge(rightPtr, leftBasePtr) + } + for (dataPtr in dataPointers) { + ctx.addPtrEdge(rightPtr, dataPtr) + } + return leftBasePtr + } + + // a.b = "str" + fun storeInstanceConst( + leftOp: JInstanceFieldRef, + rightOp: Constant, + method: SootMethod, + ) { + val leftBaseName = leftOp.base.toString() + val leftBaseType = leftOp.base.type + val sootField = leftOp.fieldRef.resolve() ?: return + val leftFieldName = sootField.name + + val leftBasePtr = pt.allocLocal(method, leftBaseName, leftBaseType) + var objs = ctx.getPointToSet(leftBasePtr) + if (objs == null) { + objs = tsp.makeNewObj(leftBaseType, leftOp, leftBasePtr) + } + for (obj in objs) { + val leftFieldPtr: PLPointer = + pt.allocObjectField(obj, leftFieldName, sootField.type, sootField) + // a.b = "str" + // "str" -> o.b flow to + tsp.addConstValue(leftFieldPtr, rightOp, method) + } + } + + // base[1] = right + fun storeArrayLocal(leftOp: JArrayRef, rightPtr: PLPointer, method: SootMethod) { + val base = leftOp.base + val leftBasePtr = pt.allocLocal(method, base.toString(), base.type) + ctx.addPtrEdge(rightPtr, leftBasePtr) + var leftBaseObjs = ctx.getPointToSet(leftBasePtr) + if (leftBaseObjs == null) { + leftBaseObjs = tsp.makeNewObj(leftOp.base.type, leftOp, leftBasePtr) + } + // propagate + for (obj in leftBaseObjs) { + val leftPtr: PLPointer = pt.allocObjectField(obj, PLUtils.DATA_FIELD, UnknownType.v()) + + // a[2] = b + // a.@data = b + // b -> o.@data + ctx.addPtrEdge(rightPtr, leftPtr) + } + } + + // left = base.field + fun loadLocalInstance( + leftOp: JimpleLocal, + rightOp: JInstanceFieldRef, + method: SootMethod + ): PLLocalPointer { + val sootField = + rightOp.field ?: throw Exception("ERROR @ loadLocalInstance $method ${rightOp.fieldRef.signature}") + val leftPtr: PLPointer = pt.allocLocal(method, leftOp.name, leftOp.type) + + val rightBase = rightOp.base as JimpleLocal + val rightBasePtr = pt.allocLocal(method, rightBase.name, rightBase.type) + var rightBaseObjs = ctx.getPointToSet(rightBasePtr) + if (rightBaseObjs == null) { + rightBaseObjs = tsp.makeNewObj(rightBase.type, leftOp, rightBasePtr) + } + //because of @data exists,may lead to java.util.ConcurrentModificationException + for (obj in HashSet(rightBaseObjs)) { + // a = b.c + // o.c -> a + // o.c flow to a + val rightPtr: PLPointer = pt.allocObjectField(obj, sootField.name, sootField.type, sootField) + + ctx.addPtrEdge(rightPtr, leftPtr) + + /* + a=b.c , + the field that b.@data points to also needs to be merged, otherwise the taint will be lost. + */ + val dataPtr = pt.allocObjectField(obj, PLUtils.DATA_FIELD, UnknownType.v()) + if (ctx.pointerFlowGraph.containsKey(dataPtr)) { + ctx.addPtrEdge(dataPtr, leftPtr) + ctx.addPtrEdge(rightBasePtr, leftPtr) + + } + } + return rightBasePtr + } + + + // $r1 = + fun assignStaticField( + leftOp: JimpleLocal, + rightOp: StaticFieldRef, + method: SootMethod, + ) { + val rightField = rightOp.field ?: return + val leftPtr: PLPointer = pt.allocLocal(method, leftOp.name, leftOp.type) + val rightPtr: PLPointer = pt.allocStaticField(rightField) + + // TODO some library class fields should be handled as objects + val declaredClass = rightField.declaringClass.toString() + if (isLibraryClass(declaredClass)) { + // $r1 = + val newRightObj = pt.allocObjectByStaticField( + rightField.type, + rightField, + rightOp, + 1 + ) + ctx.addObjToPTS(rightPtr, newRightObj) + } + ctx.addPtrEdge(rightPtr, leftPtr) + } + + //arr[2] = "test" + fun storeArrayConst(leftOp: JArrayRef, rightOp: Constant, method: SootMethod) { + val baseType = leftOp.type.arrayType.baseType + val leftBasePtr = pt.allocLocal(method, leftOp.base.toString(), leftOp.base.type) + + var leftBaseObjs = ctx.getPointToSet(leftBasePtr) + if (leftBaseObjs == null) { + leftBaseObjs = tsp.makeNewObj(leftOp.base.type, leftOp, leftBasePtr) + } + // propagate + for (obj in leftBaseObjs) { + // a[2] = "str" + // a.@data = new "str" + val leftPtr: PLPointer = pt.allocObjectField(obj, PLUtils.DATA_FIELD, UnknownType.v()) + if (baseType.toString() == "java.lang.String" || baseType.toString() == "char") { + tsp.addConstValue(leftPtr, rightOp, method) + } else { + tsp.addConstValue(leftPtr, rightOp, method) + } + } + } + + fun newInstant(leftOp: JimpleLocal, rightOp: AnyNewExpr, method: SootMethod, line: Int) { + val leftPtr: PLPointer = pt.allocLocal(method, leftOp.name, leftOp.type) + val rightObj = pt.allocObject(rightOp.type, method, rightOp, line) + // PLLog.logErr("NewInstant "+leftPtr+" -> "+rightObj); + ctx.addObjToPTS(leftPtr, rightObj) + } + + // r1 = "str" + // A.a = "str" + fun storeLocalOrStaticFieldConst(leftOp: Value, rightOp: Constant, method: SootMethod) { + val leftPtr: PLPointer? + if (leftOp is JimpleLocal) { + val localName = leftOp.name + leftPtr = pt.allocLocal(method, localName, leftOp.getType()) + } else if (leftOp is StaticFieldRef) { + leftPtr = pt.allocStaticField(leftOp.field) + } else { + throw Exception("unknown $leftOp=$rightOp") + } + tsp.addConstValue(leftPtr, rightOp, method) + } + + fun stmtReturn( + recvPtr: PLLocalPointer?, + calleeReturnStmt: JReturnStmt, + callee: SootMethod, + ) { + val retOp = calleeReturnStmt.op + if (retOp is JimpleLocal) { + val retPtr: PLPointer = pt.allocLocal(callee, retOp.name, retOp.type) + tsp.addReturnPtrMap(callee, retPtr) + if (recvPtr != null) { + ctx.addPtrEdge(retPtr, recvPtr) + } + } else { // Constant + // NumericConstant,StringConstant,NullConstant,ClassConstant + if (recvPtr != null) { + val constPtr = tsp.addConstValue(recvPtr, retOp as Constant, callee) + tsp.addReturnPtrMap(callee, constPtr) + } else { + val constPtr = pt.allocLocal(callee, PLUtils.constSig(retOp as Constant), retOp.type) + tsp.addReturnPtrMap(callee, constPtr) + } + } + } + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/taintflow/TaintAnalyzer.kt b/src/main/kotlin/net/bytedance/security/app/taintflow/TaintAnalyzer.kt new file mode 100644 index 0000000..fbdf2c9 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/taintflow/TaintAnalyzer.kt @@ -0,0 +1,155 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.taintflow + +import net.bytedance.security.app.pointer.PLLocalPointer +import net.bytedance.security.app.pointer.PLPointer +import net.bytedance.security.app.rules.TaintFlowRule +import net.bytedance.security.app.util.profiler +import soot.SootMethod +import soot.Type +import soot.jimple.Stmt + +/** +TaintAnalyzerData represents the parsed source and sink in Rule. + */ +class TaintAnalyzerData { + + + /** + *key is the PLLocalPointer's name, value is the PLLocalPointer. + * it contains all sources and sinks + */ + + var pointerIndexMap: MutableMap = HashMap() + + // sources + var sourcePointerSet: MutableSet = HashSet() + + var sinkPointerSet: MutableSet = HashSet() + + // the stmt which contains the sources and sinks + var ptrStmtMapSrcSink: MutableMap> = HashMap() + + + private fun allocPtr(method: SootMethod, localName: String, origType: Type): PLLocalPointer { + val ptrSig = PLLocalPointer.getLocalLongSignature(method, localName) + if (pointerIndexMap.containsKey(ptrSig)) { + return pointerIndexMap[ptrSig] as PLLocalPointer + } + val ptr = PLLocalPointer(method, localName, origType) + pointerIndexMap[ptrSig] = ptr + return ptr + } + + fun allocSourcePtr(method: SootMethod, localName: String, origType: Type): PLLocalPointer { + val ptr = allocPtr(method, localName, origType) + sourcePointerSet.add(ptr) + return ptr + } + + private fun allocSinkPtr(method: SootMethod, localName: String, origType: Type): PLLocalPointer { + val ptr = allocPtr(method, localName, origType) + sinkPointerSet.add(ptr) + return ptr + } + + + fun allocPtrWithStmt( + stmt: Stmt, + method: SootMethod, + localName: String, + origType: Type, + isSource: Boolean + ): PLLocalPointer { + val ptr = if (isSource) allocSourcePtr(method, localName, origType) + else { + allocSinkPtr(method, localName, origType) + } + if (!ptrStmtMapSrcSink.containsKey(ptr)) { + ptrStmtMapSrcSink[ptr] = HashSet() + } + ptrStmtMapSrcSink[ptr]!!.add(stmt) + return ptr + } + +} + +class TaintAnalyzer { + var data: TaintAnalyzerData = TaintAnalyzerData() + val rule: TaintFlowRule + val entryMethod: SootMethod + + //analyze depth for this analyzer + val thisDepth: Int + val sinkPtrSet get() = this.data.sinkPointerSet + val sourcePtrSet get() = this.data.sourcePointerSet + + constructor(rule: TaintFlowRule, entryMethod: SootMethod) { + this.rule = rule + this.entryMethod = entryMethod + this.thisDepth = rule.traceDepth + } + + constructor(rule: TaintFlowRule, entryMethod: SootMethod, data: TaintAnalyzerData) { + this.rule = rule + this.entryMethod = entryMethod + this.data = data + this.thisDepth = rule.traceDepth + } + + constructor( + rule: TaintFlowRule, + entryMethod: SootMethod, + data: TaintAnalyzerData, + srcPtr: PLLocalPointer, + sinkPtr: PLLocalPointer, + thisDepth: Int + ) { + this.rule = rule + this.entryMethod = entryMethod + this.data.sourcePointerSet.add(srcPtr) + this.data.sinkPointerSet.add(sinkPtr) + this.data.pointerIndexMap[srcPtr.signature()] = srcPtr + this.data.pointerIndexMap[sinkPtr.signature()] = sinkPtr + data.ptrStmtMapSrcSink[srcPtr]?.let { + this.data.ptrStmtMapSrcSink[srcPtr] = it + } + data.ptrStmtMapSrcSink[sinkPtr]?.let { + this.data.ptrStmtMapSrcSink[sinkPtr] = it + } + this.thisDepth = thisDepth + } + + init { + profiler.addTaintAnalyzerCount() + } + + + fun dump(): String { + return """ + rule=${rule.name}, + EntryMethod=${entryMethod.signature}, + ptrIndexMap=${data.pointerIndexMap}, + sourcePtrSet=${data.sourcePointerSet}, + sinkPtrSet=${data.sinkPointerSet}, + ptrStmtMapSrcSink=${data.ptrStmtMapSrcSink} + """.trimIndent() + } + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/taintflow/TaintTweakTaintFlowRule.kt b/src/main/kotlin/net/bytedance/security/app/taintflow/TaintTweakTaintFlowRule.kt new file mode 100644 index 0000000..d511a4a --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/taintflow/TaintTweakTaintFlowRule.kt @@ -0,0 +1,55 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.taintflow + +import net.bytedance.security.app.TaintTweakData +import net.bytedance.security.app.engineconfig.VariableFlowConfig +import soot.SootMethod + +/** + * TaintTweak in rule file + */ +class TaintTweakTaintFlowRule(tw: TaintTweakData, private val defaultTaintFlowRule: DefaultVariableFlowRule) : + IVariableFlowRule { + + var methodRule: Map> = HashMap() + + init { + + tw.MethodName?.mapValues { + VariableFlowConfig.parseListRule(it.value).toFlowItem() + }?.let { + this.methodRule += it + } + + tw.MethodSignature?.mapValues { VariableFlowConfig.parseListRule(it.value).toFlowItem() }?.let { + this.methodRule += it + } + + } + + override fun flow(callerMethod: SootMethod, calleeMethod: SootMethod): List { + methodRule[calleeMethod.signature]?.let { + return it + } + methodRule[calleeMethod.name]?.let { + return it + } + return defaultTaintFlowRule.flow(callerMethod, calleeMethod) + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/taintflow/TwoStagePointerAnalyze.kt b/src/main/kotlin/net/bytedance/security/app/taintflow/TwoStagePointerAnalyze.kt new file mode 100644 index 0000000..b103b05 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/taintflow/TwoStagePointerAnalyze.kt @@ -0,0 +1,859 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.taintflow + +import kotlinx.coroutines.* +import net.bytedance.security.app.Log +import net.bytedance.security.app.PLUtils +import net.bytedance.security.app.engineconfig.isLibraryClass +import net.bytedance.security.app.pointer.PLLocalPointer +import net.bytedance.security.app.pointer.PLObject +import net.bytedance.security.app.pointer.PLPointer +import net.bytedance.security.app.pointer.PointerFactory +import net.bytedance.security.app.util.profiler +import net.bytedance.security.app.util.runInMilliSeconds +import net.bytedance.security.app.util.toSortedMap +import net.bytedance.security.app.util.toSortedSet +import soot.* +import soot.jimple.* +import soot.jimple.internal.* + +class TwoStagePointerAnalyze( + val name: String, //for debug, + val entryMethod: SootMethod, + val ctx: AnalyzeContext, + val traceDepth: Int, + private val pointerPropagationRule: IPointerFlowRule, + private val taintFlowRule: IVariableFlowRule, + private val methodAnalyzeMode: IMethodAnalyzeMode, + private val analyzeTimeInMilliSeconds: Long, +) { + val pt: PointerFactory get() = ctx.pt + private val st = StmtTransfer(ctx, pt, this) + private val orh = ObscureRuleHandler(ctx, pt) + + + //key is a method call site,value are callee functions. + private var callGraph: MutableMap> = HashMap() + private var thisStageStart: Long = 0 + + + private var patchedMethods: MutableSet = HashSet() + + private var returnPointerMap: MutableMap> = HashMap() + + /* + The following three maps are specifically used to analyze method calls like obj.func(A1, A2, A3). + stmtMethodMap: key is the stmt,value is the caller method + stmtPtrMap: key is the stmt, value is obj(this) + stmtObjMap:key is the stmt, value is the object set that this points to + */ + + private var stmtMethodMap: MutableMap = HashMap() + private var stmtPtrMap: MutableMap = HashMap() + private var stmtObjMap: MutableMap> = HashMap() + + + private var pseudoObjectOrFieldIndex = 0 + + init { + profiler.addTwoStagePointerAnalyzeCount() + } + + private var scope: CoroutineScope? = null + private fun canContinueAnalyze(): Boolean { + return scope!!.isActive + } + + /** + The entrance of taint analysis and pointer analysis + */ + suspend fun doPointerAnalyze() { + val localScope = CoroutineScope(Dispatchers.Default) + profiler.startPointAnalyze(name) + thisStageStart = System.currentTimeMillis() + try { + val job = localScope.launch(Dispatchers.Default) { + scope = this + analyzeMethod(entryMethod, null, 0) + } + runInMilliSeconds(job, analyzeTimeInMilliSeconds, "$name-step1") {} + } catch (ex: Exception) { + ex.printStackTrace() + } + yield() + Log.logInfo("$name fistStageAnalyze finished") + thisStageStart = System.currentTimeMillis() + try { + val job = localScope.launch(Dispatchers.Default) { + scope = this + secondStageAnalyze() + } +// job.join() + runInMilliSeconds(job, analyzeTimeInMilliSeconds, "$name-step2") {} + } catch (ex: Exception) { + ex.printStackTrace() + } + profiler.stopPointAnalyze(name) + Log.logInfo("$name secondStageAnalyze finished, \nbaseInfo=${ctx.baseInfo()}") + } + + /** + * process stmt like r=a.f(a1,a2,a3) + * @param sootMethod the method f + * @param recvPtr r if exists + * @param curTraceDepth the depth of the call stack + * todo add @return to simplify the process,for example: + * r0=a.f() a.r1->a.@return a.r2->a.@return a.@return ->caller.r0 + */ + private suspend fun analyzeMethod( + sootMethod: SootMethod, recvPtr: PLLocalPointer?, curTraceDepth: Int + ): Boolean { + if (curTraceDepth > traceDepth) { + return true + } + yield() + if (!canContinueAnalyze()) { + return false + } + var line = 0 + for (unit in sootMethod.activeBody.units) { + line++ + val stmt = unit as Stmt + if (!canContinueAnalyze()) { + return false + } + analyzeStmtInterProcedure(stmt, sootMethod, recvPtr, line, curTraceDepth + 1) + } + return false + } + + + private suspend fun analyzeStmtInterProcedure( + stmt: Stmt, method: SootMethod, recvPtr: PLLocalPointer?, line: Int, curTraceDepth: Int + ) { + if (stmt is JInvokeStmt) { + val invokeExpr = stmt.getInvokeExpr() + val callSite = createCallSite(method, line) + if (invokeExpr is JStaticInvokeExpr || invokeExpr is JDynamicInvokeExpr) { + staticInvoke( + stmt, method, callSite, null, invokeExpr as AbstractInvokeExpr, line, curTraceDepth + ) + } else if (invokeExpr is InstanceInvokeExpr) { + val basePtr = instanceInvoke(stmt, method, callSite, null, invokeExpr, line, curTraceDepth) + basePtr?.let { addToFixPointAlgoCache(it, method, stmt) } + } + } else if (stmt is JIdentityStmt) { + // l2 := @parameter1: int + st.identityStmt(stmt, method) + } else if (stmt is JAssignStmt) { + val leftExpr = stmt.leftOp + when (val rightExpr = stmt.rightOp) { + is JStaticInvokeExpr -> { + val callSite = createCallSite(method, line) + val localCallRecv = leftExpr as JimpleLocal + val localRecvPtr = pt.allocLocal(method, localCallRecv.name, localCallRecv.type) + staticInvoke( + stmt, method, callSite, localRecvPtr, rightExpr, line, curTraceDepth + ) + } + is JDynamicInvokeExpr -> { + val callSite = createCallSite(method, line) + val localCallRecv = leftExpr as JimpleLocal + val localRecvPtr = pt.allocLocal(method, localCallRecv.name, localCallRecv.type) + staticInvoke( + stmt, method, callSite, localRecvPtr, rightExpr, line, curTraceDepth + ) + } + is InstanceInvokeExpr -> { + val callSite = createCallSite(method, line) + val recvOp = leftExpr as JimpleLocal + val localRecvPtr = pt.allocLocal(method, recvOp.name, recvOp.type) + val basePtr = instanceInvoke(stmt, method, callSite, localRecvPtr, rightExpr, line, curTraceDepth) + basePtr?.let { addToFixPointAlgoCache(it, method, stmt) } + } + is AbstractBinopExpr -> { + // a = 3 + 4 + // a = b + c + // a = A.b + 3 + // a = 3 + b + st.binaryOp(leftExpr as JimpleLocal, rightExpr, method) + } + is UnopExpr -> { + // $i1 = lengthof $r1 + // $i1 = neg $i0 + st.unaryOp(leftExpr as JimpleLocal, rightExpr, method) + } + is JCastExpr -> { + // a = (A)b + // a = (int)4 + st.castExpr(leftExpr as JimpleLocal, rightExpr, method) + } + is JArrayRef -> { + // 'a = b[2]' as load + st.loadArray(leftExpr as JimpleLocal, rightExpr, method) + } + is JimpleLocal -> { + val rightPtr = pt.allocLocal(method, rightExpr.name, rightExpr.getType()) + when (leftExpr) { + is JInstanceFieldRef -> { + // a.b = c + val basePtr = st.storeInstanceLocal(leftExpr, rightPtr, method) + addToFixPointAlgoCache(basePtr, method, stmt) + } + is StaticFieldRef -> { + // A.b = c + val leftPtr = pt.allocStaticField(leftExpr.field) + if (rightExpr.getType() is ArrayType && ctx.pointerToObjectSet.containsKey(rightPtr)) { + val objs = + HashSet(ctx.getPointToSet(rightPtr)!!) //copy to avoid ConcurrentModificationException + for (obj in objs) { + val objPtr = pt.allocObjectField(obj, PLUtils.DATA_FIELD, UnknownType.v()) + ctx.addPtrEdge(objPtr, rightPtr) + } + } + ctx.addPtrEdge(rightPtr, leftPtr) + } + is JimpleLocal -> { + // a = c + val leftPtr = pt.allocLocal(method, leftExpr.name, leftExpr.getType()) + ctx.addPtrEdge(rightPtr, leftPtr) + } + else -> { // JArrayRef + // arr[1] = c + st.storeArrayLocal(leftExpr as JArrayRef, rightPtr, method) + } + } + } + is JInstanceFieldRef -> { + // a = c.b + val basePtr = st.loadLocalInstance(leftExpr as JimpleLocal, rightExpr, method) + addToFixPointAlgoCache(basePtr, method, stmt) + } + is StaticFieldRef -> { + // a = A.b + st.assignStaticField(leftExpr as JimpleLocal, rightExpr, method) + } + is AnyNewExpr -> { + // a = new A() + st.newInstant(leftExpr as JimpleLocal, rightExpr, method, line) + } + is Constant -> { // StringConstant,NullConstant,ClassConstant,NumericConstant + when (leftExpr) { + is JInstanceFieldRef -> { + // a.b = "33" + st.storeInstanceConst(leftExpr, rightExpr, method) + } + is JArrayRef -> { + // arr[2] = "test" as store + st.storeArrayConst(leftExpr, rightExpr, method) + } + is StaticFieldRef, is JimpleLocal -> { + // a = "str" + // A.a = "str" + st.storeLocalOrStaticFieldConst(leftExpr, rightExpr, method) + } + else -> { + throw Exception("unknown stmt=$stmt") + } + } + } + } + } else if (stmt is JReturnStmt) { + st.stmtReturn(recvPtr, stmt, method) + } + } + + + private suspend fun staticInvoke( + stmt: Stmt, + caller: SootMethod, + callSite: String, + recvPtr: PLLocalPointer?, + invokeExpr: AbstractInvokeExpr, + line: Int, + curTraceDepth: Int + ) { + if (!canContinueAnalyze()) { + return + } + val mode = methodAnalyzeMode.methodMode(invokeExpr.method) + if (mode == MethodAnalyzeMode.Skip) { + return + } + val callee = invokeExpr.method + if (!checkAddCallGraph(callSite, callee)) { + transferParams(invokeExpr, caller, callee) + if (!checkAddRM(callee)) { + if (mode == MethodAnalyzeMode.Obscure) { + patchedMethods.add(callee) + patchMethod(stmt, caller, callee, null, recvPtr, line) + } else { + val isReachedMax = analyzeMethod(callee, recvPtr, curTraceDepth) + if (isReachedMax) { + removeRM(callee) + } + } + } else { + if (recvPtr != null) { + if (patchedMethods.contains(callee)) { + patchMethod(stmt, caller, callee, null, recvPtr, line) + } else { + val retPtrSet = returnPointerMap[callee] + if (retPtrSet != null && retPtrSet.isNotEmpty()) { + for (retPtr in retPtrSet) { + ctx.addPtrEdge(retPtr, recvPtr) + } + } + } + } + } + } + } + + /** + * ret=obj.f(arg1,arg2) + * @param stmt ret=obj.f(arg1,arg2) + * @param caller the method that contains the [stmt] + * @param callSite call site + * @param recvPtr ret if exists + * @param invokeExpr obj.f(arg1,arg2) + * @param line: stmt index + * @param curTraceDepth: max depth + */ + private suspend fun instanceInvoke( + stmt: Stmt, + caller: SootMethod, + callSite: String, + recvPtr: PLLocalPointer?, + invokeExpr: InstanceInvokeExpr, + line: Int, + curTraceDepth: Int + ): PLLocalPointer? { + if (!canContinueAnalyze()) { + return null + } + val mode = methodAnalyzeMode.methodMode(invokeExpr.method) + if (mode == MethodAnalyzeMode.Skip) { + return null + } + // base.func() + val baseName = invokeExpr.base.toString() + val baseType = invokeExpr.base.type + val basePtr = pt.allocLocal(caller, baseName, baseType) + val baseObjs = ctx.getPointToSet(basePtr) + val typeObjs: MutableSet = HashSet() + + if (baseObjs != null) { + for (obj in baseObjs) { + typeObjs.add(obj) + } + } + + if (typeObjs.size == 1) { + val oneObj = typeObjs.iterator().next() + val objType = oneObj.classType + if (objType !is ArrayType) { + val sc = Scene.v().getSootClassUnsafe(objType.toString(), false) + if (sc != null && sc.isInterface && !isLibraryClass(sc.name)) { + val subClassSet = HashSet() + PLUtils.getAllSubCLass(sc, subClassSet) + for (newClass in subClassSet) { + if (isLibraryClass(newClass.name) || newClass.isInterface || newClass.isPhantom) { + continue + } + var isMatch = false + for (sm in newClass.methods) { + if (sm.subSignature == invokeExpr.methodRef.subSignature.toString()) { + isMatch = true + break + } + } + if (isMatch) { + val newObj = pt.allocObject( + newClass.type, getPseudoEntryMethod(), null, pseudoObjectOrFieldIndex++ + ) + typeObjs.add(newObj) + } + } + } + } else { +// throw Exception("$oneObj ${objType} is ArrayType") + } + } + var typeObjs2: Set = typeObjs + if (typeObjs.isEmpty()) { + typeObjs2 = makeNewObj(baseType, invokeExpr, basePtr) + } + for (typeObj in typeObjs2) { + val callee = dispatchInstanceCall(invokeExpr, typeObj) ?: continue + handleInstanceInvoke( + stmt, callee, baseType, basePtr, caller, callSite, recvPtr, invokeExpr, line, curTraceDepth + ) + } + return basePtr + } + + // a.f(a,b,c) + // a.b=c + //c =a.b + private fun addToFixPointAlgoCache(basePtr: PLLocalPointer, method: SootMethod, stmt: Stmt) { + if (!ctx.pointerToObjectSet.containsKey(basePtr)) { + return + } + val allBaseObjs = ctx.getPointToSet(basePtr) + val handledObjs = allBaseObjs?.toHashSet() ?: HashSet() + stmtMethodMap[stmt] = method + stmtObjMap[stmt] = handledObjs + stmtPtrMap[stmt] = basePtr + } + + + fun addReturnPtrMap(callee: SootMethod, retPtr: PLPointer) { + var retPtrSet = returnPointerMap[callee] + if (retPtrSet == null) { + retPtrSet = HashSet() + returnPointerMap[callee] = retPtrSet + } + retPtrSet.add(retPtr) + } + + fun makeNewObj(type: Type, v: Value, ptr: PLPointer): Set { + val newObj = pt.allocObject(type, getPseudoEntryMethod(), v, pseudoObjectOrFieldIndex++) + val objs = setOf(newObj) + ctx.addObjToPTS(ptr, newObj) + return objs + } + + + /** + * r1="str", + * @param nextPtr r1 + * @param constant "str" + * @param constMethod method which contains this stmt(r1="str") + */ + fun addConstValue( + nextPtr: PLPointer, + constant: Constant, + constMethod: SootMethod, + ): PLLocalPointer { + val constPtr = pt.allocLocal(constMethod, PLUtils.constSig(constant), constant.type) + constPtr.setConst(constant) + ctx.addPtrEdge(constPtr, nextPtr) + return constPtr + } + + + // a = b.c + // b.c=a + private fun loadOrStoreFixPointAlgo( + local: JimpleLocal, field: JInstanceFieldRef, method: SootMethod, typeObjs: Set, isLoad: Boolean + ) { + val sootField = field.field + val fieldName = sootField.name + val base = field.base as JimpleLocal + val basePtr = pt.allocLocal(method, base.name, base.type) + val localPtr: PLPointer = pt.allocLocal(method, local.name, local.type) + for (obj in typeObjs) { + if (!canContinueAnalyze()) { + return + } + val fieldObjPtr: PLPointer = pt.allocObjectField(obj, fieldName, sootField.type, sootField) + val dataPtr = pt.allocObjectField(obj, PLUtils.DATA_FIELD, UnknownType.v()) + recordMethodTakesTime("fieldObjPtr=${fieldObjPtr.signature()},localPtr=${localPtr.signature()}") { + if (isLoad) { + ctx.addPtrEdge(fieldObjPtr, localPtr) + if (ctx.pointerFlowGraph.containsKey(dataPtr)) { + ctx.addPtrEdge(dataPtr, localPtr) + ctx.addPtrEdge(basePtr, localPtr) + } + } else { + ctx.addPtrEdge(localPtr, fieldObjPtr) + if (ctx.pointerFlowGraph.containsKey(dataPtr)) { + ctx.addPtrEdge(localPtr, dataPtr) + ctx.addPtrEdge(localPtr, basePtr) + } + } + } + } + } + + + private fun transferParams( + invokeExpr: InvokeExpr, + caller: SootMethod, + callee: SootMethod, + ) { + for ((argIndex, i) in (0 until invokeExpr.argCount).withIndex()) { + val arg = invokeExpr.getArg(i) + val paramType = invokeExpr.methodRef.getParameterType(i) + if (arg is Constant) { // NumericConstant,StringConstant,NullConstant,ClassConstant + val dstPtr: PLPointer = pt.allocLocal(callee, PLUtils.PARAM + argIndex, paramType) + addConstValue(dstPtr, arg, caller) + } else { + val local = arg as JimpleLocal + val srcPtr: PLPointer = pt.allocLocal(caller, local.name, local.type) + val dstPtr: PLPointer = pt.allocLocal(callee, PLUtils.PARAM + argIndex, paramType) + ctx.addPtrEdge(srcPtr, dstPtr) + } + } + } + + + private fun dispatchInstanceCall(invokeExpr: InstanceInvokeExpr, obj: PLObject): SootMethod? { + // check they satisfied the hierarchy. + if (!Scene.v().orMakeFastHierarchy.canStoreType(obj.classType, invokeExpr.base.type)) { + return null + } + if (obj.classType is ArrayType) { + return invokeExpr.method + } + // class of object + var objClass = Scene.v().getSootClass(obj.classType.toString()) ?: return null + val calleeClass = invokeExpr.methodRef.declaringClass + + if (invokeExpr is JSpecialInvokeExpr) { + objClass = calleeClass + } + val methodSubSig = invokeExpr.methodRef.subSignature.toString() + + + if (isLibraryClass(objClass.name)) { + return PLUtils.dispatchCall(calleeClass, methodSubSig) + } + return PLUtils.dispatchCall(objClass, methodSubSig) + + } + + /** + * for method's obscure analyze mode, + * except user's specified obscure mode,there are other two case: + * 1. exceed the max depth + * 2. cannot find method body (for example ,cannot find any implementation for an interface) + * @param stmt where this method call occurs + * @param caller where this stmt in + * @param calleeMethod the callee method in stmt + * @param basePtr calleeMethod's this pointer if calleeMethod is static, otherwise it's null + * @param recvPtr c=a.f() if c exists + * @param line callsite + */ + private fun patchMethod( + stmt: Stmt, caller: SootMethod, calleeMethod: SootMethod, + basePtr: PLLocalPointer?, recvPtr: PLLocalPointer?, line: Int + ) { + var basePtr2 = basePtr + //disallow flow taint to Context + if (basePtr2 != null && isContextPtr(basePtr2)) { + basePtr2 = null + } + if (recvPtr != null) { + val newObj = pt.allocObject(recvPtr.ptrType, caller, stmt.invokeExpr, line) + ctx.addObjToPTS(recvPtr, newObj) + } + + var baseDataPtrs: MutableSet? = null + if (basePtr2 != null) { + baseDataPtrs = HashSet() + for (baseObj in ctx.getPointToSet(basePtr2)!!) { + val baseDataPtr: PLPointer = pt.allocObjectField(baseObj, PLUtils.DATA_FIELD, UnknownType.v()) + baseDataPtrs.add(baseDataPtr) + } +// } + } + val argPtrs: MutableList = ArrayList() + if (calleeMethod.parameterCount > 0) { + val invokeExpr = stmt.invokeExpr + for (i in 0 until invokeExpr.argCount) { + val arg = invokeExpr.getArg(i) + if (arg is Constant) { // NumericConstant,StringConstant,NullConstant,ClassConstant + val paramPtr = pt.allocLocal(calleeMethod, PLUtils.PARAM + i, arg.getType()) + val constPtr = addConstValue(paramPtr, arg, caller) + argPtrs.add(constPtr) + // PLLog.logErr("add edge "+constPtr +" -> "+paramPtr); + } else { + val argName = (arg as JimpleLocal).name + val argPtr = pt.allocLocal(caller, argName, arg.getType()) + argPtrs.add(argPtr) + } + } + } + pointerPropagationRule.flow(calleeMethod)?.let { + orh.addEdgeByRule(stmt, it, basePtr2, baseDataPtrs, recvPtr, argPtrs, true) + } + taintFlowRule.flow(caller, calleeMethod).let { + orh.addEdgeByRule(stmt, it, basePtr2, baseDataPtrs, recvPtr, argPtrs, false) + } + } + + + private suspend fun handleInstanceInvoke( + stmt: Stmt, + callee: SootMethod, + baseType: Type, + basePtr: PLLocalPointer, + caller: SootMethod, + callSite: String, + recvPtr: PLLocalPointer?, + invokeExpr: InstanceInvokeExpr, + line: Int, + curTraceDepth: Int, + processNewMethod: Boolean = true, + ) { + if (!canContinueAnalyze()) { + return + } + if (!checkAddCallGraph(callSite, callee)) { + val thisPtr: PLPointer = pt.allocLocal(callee, PLUtils.THIS_FIELD, baseType) + ctx.addPtrEdge(basePtr, thisPtr) + transferParams(invokeExpr, caller, callee) + + if (!checkAddRM(callee)) { + if (!processNewMethod) { + return + } + val mode = methodAnalyzeMode.methodMode(callee) + if (mode == MethodAnalyzeMode.Obscure) { + patchMethod(stmt, caller, callee, basePtr, recvPtr, line) + patchedMethods.add(callee) + } else { + val isReachedMax = analyzeMethod(callee, recvPtr, curTraceDepth) + if (isReachedMax) { + removeRM(callee) + } + } + } else { + if (patchedMethods.contains(callee)) { + patchMethod(stmt, caller, callee, basePtr, recvPtr, line) + } else { + if (recvPtr != null) { + val retPtrSet = returnPointerMap[callee] + if (retPtrSet != null && retPtrSet.isNotEmpty()) { + for (retPtr in retPtrSet) { + ctx.addPtrEdge(retPtr, recvPtr) + } + } + } // if(callerRecv != null){ + } // + } + } + } + + + private suspend fun secondStageAnalyze() { + val methodSigArr: MutableList = ArrayList() + val stmtArr: MutableList = ArrayList() + val ptrArr: MutableList = ArrayList() + val curObjArr: MutableList> = ArrayList() + while (true) { + yield() + if (!canContinueAnalyze()) { + return + } + methodSigArr.clear() + stmtArr.clear() + ptrArr.clear() + curObjArr.clear() + val fixedPoint = isReachedFixPoint(methodSigArr, stmtArr, ptrArr, curObjArr) + if (fixedPoint) { + break + } + for (i in methodSigArr.indices) { + if (!canContinueAnalyze()) { + return + } + fixPointAlgoWithObjs(methodSigArr[i], stmtArr[i], ptrArr[i], curObjArr[i]) + } + } + } + + private suspend fun fixPointAlgoWithObjs( + caller: SootMethod, + stmt: Stmt, + basePtr: PLLocalPointer, + typeObjs: Set + ) { + if (stmt.containsInvokeExpr()) { + instanceInvokeWithObjs(caller, stmt, basePtr, typeObjs) + } else if (stmt is JAssignStmt) { + val leftOp = stmt.leftOp + val rightOp = stmt.rightOp + if (leftOp is JimpleLocal) { + val fieldRef = rightOp as JInstanceFieldRef + loadOrStoreFixPointAlgo(leftOp, fieldRef, basePtr.method, typeObjs, true) + } else { + val local = rightOp as JimpleLocal + val fieldRef = leftOp as JInstanceFieldRef + loadOrStoreFixPointAlgo(local, fieldRef, basePtr.method, typeObjs, false) + } + } + } + + private suspend fun instanceInvokeWithObjs( + caller: SootMethod, stmt: Stmt, basePtr: PLLocalPointer, typeObjs: Set + ): PLLocalPointer { + var recvPtr: PLLocalPointer? = null + val invokeExpr: InstanceInvokeExpr + if (stmt is JAssignStmt) { + val callerRecv = stmt.leftOp as JimpleLocal + recvPtr = pt.allocLocal(caller, callerRecv.name, callerRecv.type) + invokeExpr = stmt.rightOp as InstanceInvokeExpr + } else { + invokeExpr = stmt.invokeExpr as InstanceInvokeExpr + } + val callSite = createCallSite(caller, stmt.javaSourceStartLineNumber) + for (typeObj in typeObjs) { + // below is the dispatch + val callee = dispatchInstanceCall(invokeExpr, typeObj) ?: continue + var curTraceDepth = 0 + if (traceDepth > 6) { + curTraceDepth = traceDepth - 6 + } + recordMethodTakesTime("$callSite:$callee") { + handleInstanceInvoke( + stmt, + callee, + basePtr.ptrType, + basePtr, + caller, + callSite, + recvPtr, + invokeExpr, + stmt.javaSourceStartLineNumber, + curTraceDepth, + false + ) + } + } + return basePtr + } + + + private fun isReachedFixPoint( + methods: MutableList, + stmtArr: MutableList, + ptrArr: MutableList, + curObjArr: MutableList> + ): Boolean { + for ((stmt, basePtr) in stmtPtrMap) { + if (!stmtMethodMap.containsKey(stmt)) { + continue + } + val method = stmtMethodMap[stmt]!! + val curObjSet: MutableSet = ctx.getPointToSet(basePtr)?.toHashSet() ?: HashSet() + val handledObjSet = stmtObjMap[stmt]!! + curObjSet.removeAll(handledObjSet) + handledObjSet.addAll(curObjSet) + val iterator = curObjSet.iterator() + while (iterator.hasNext()) { + val obj = iterator.next() + if (obj.isPseudoObj) { + iterator.remove() + } + } + if (curObjSet.isEmpty()) { + continue + } + + methods.add(method) + stmtArr.add(stmt) + ptrArr.add(basePtr) + curObjArr.add(curObjSet) + } + return methods.isEmpty() + } + + + private fun createCallSite(caller: SootMethod, line: Int): String { + return "$caller:$line" + } + + private fun checkAddCallGraph(callSite: String, callee: SootMethod): Boolean { + var isInCG = true + var dstSet = callGraph[callSite] + if (dstSet == null) { + dstSet = HashSet() + callGraph[callSite] = dstSet + } + if (!dstSet.contains(callee)) { + isInCG = false + dstSet.add(callee) + } + return isInCG + } + + private fun removeRM(method: SootMethod) { + ctx.rm.remove(method) + } + + private fun checkAddRM(methodSig: SootMethod): Boolean { + var isInRM = true + if (!ctx.rm.contains(methodSig)) { + isInRM = false + ctx.rm.add(methodSig) + } + return isInRM + } + + @Suppress("unused") + fun dump(): String { + var s = """ + entryMethod=${entryMethod.signature}, + patchedMethods= ${patchedMethods.toSortedSet()}, + ReturnPtrMap=${returnPointerMap.toSortedMap()}, + stmtMethodMap=${stmtMethodMap.toSortedMap()}, + stmtPtrMap=${stmtPtrMap.toSortedMap()}, + stmtObjMap=${stmtObjMap.toSortedMap()}, + ptrIndexMap=${pt.ptrIndexMap.keys.toSortedSet()}, + objIndexMap=${pt.objIndexMap.keys.toSortedSet()}, + """.trimIndent() + s += "PLContext={\n${ctx.dump()}\n}\n" + return s + } + + companion object { + private var pseudoMethod: SootMethod? = null + + @Synchronized + fun getPseudoEntryMethod(): SootMethod { + if (pseudoMethod == null) { + var clz = Scene.v().getSootClass(PLUtils.CUSTOM_CLASS) + if (clz.isPhantom) { + PLUtils.createCustomClass() + clz = Scene.v().getSootClass(PLUtils.CUSTOM_CLASS) + } + pseudoMethod = PLUtils.entryMethod(clz, listOf(), false) + } + return pseudoMethod!! + } + + inline fun recordMethodTakesTime(name: String, defaultTime: Int = 20000, action: () -> Unit) { + val start = System.currentTimeMillis() + action() + val end = System.currentTimeMillis() + if (end - start > defaultTime) { + Log.logWarn("recordMethodTakesTime $name takes ${end - start}ms") + } + } + + private fun isContextPtr(ptr: PLPointer): Boolean { + val type = ptr.ptrType + return type.toString() == "android.content.Context" + } + } + +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/ui/ConstExtractModeHtmlWriter.kt b/src/main/kotlin/net/bytedance/security/app/ui/ConstExtractModeHtmlWriter.kt new file mode 100644 index 0000000..dbf109f --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/ui/ConstExtractModeHtmlWriter.kt @@ -0,0 +1,86 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.ui + +import kotlinx.html.TagConsumer +import kotlinx.html.a +import kotlinx.html.classes +import net.bytedance.security.app.Log +import net.bytedance.security.app.result.IVulnerability +import net.bytedance.security.app.result.OutputSecResults +import net.bytedance.security.app.result.VulnerabilityItem +import net.bytedance.security.app.rules.IRule +import soot.SootMethod +import soot.jimple.Stmt + +/** + * output for ConstStringMode and ConstNumberMode + * @param constStr Matched string or number + * @param stmt The statement that accesses the constant constStr + * @param method The method which stmt belongs to + */ +class ConstExtractModeHtmlWriter( + private val secResults: OutputSecResults, + private val rule: IRule, + val method: SootMethod, + val stmt: Stmt, + private val constStr: String +) : HtmlWriter(rule.desc), AddVulnerabilityAndSaveResult { + + override fun genContent(tag: TagConsumer<*>) { + genVulInfo(tag) + tag.a { + classes = setOf(classVulnerabilityDetail) + +constStr + } + genMethodWithHighlight(tag, method, setOf(stmt)) + genMethodJavaSource(tag, method) + } + + override suspend fun addVulnerabilityAndSaveResultToOutput() { + val stringList: MutableList = ArrayList() + val apiSig = stmt.invokeExpr.methodRef.signature + stringList.add(method.signature) + stringList.add(apiSig) + stringList.add(constStr) + val tosUrl = saveContent(generateHtml(), htmlName) + Log.logDebug("Write Vulnerability to $tosUrl") + secResults.addOneVulnerability( + VulnerabilityItem( + rule, + tosUrl, + ConstExtractModeVulnerability(stringList, method.signature, constStr) + ) + ) + } +} + +class ConstExtractModeVulnerability( + override val target: List, + override val position: String, + val constValue: String +) : + IVulnerability { + override fun toDetail(): Map { + return mapOf( + "target" to target, + "ConstValue" to constValue, + "position" to position + ) + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/ui/FindClassModeHtmlWriter.kt b/src/main/kotlin/net/bytedance/security/app/ui/FindClassModeHtmlWriter.kt new file mode 100644 index 0000000..49d4f71 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/ui/FindClassModeHtmlWriter.kt @@ -0,0 +1,104 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.ui + +import kotlinx.html.TagConsumer +import kotlinx.html.a +import kotlinx.html.classes +import kotlinx.html.div +import net.bytedance.security.app.preprocess.CallSite +import net.bytedance.security.app.result.IVulnerability +import net.bytedance.security.app.result.OutputSecResults +import net.bytedance.security.app.result.VulnerabilityItem +import net.bytedance.security.app.rules.IRule +import soot.SootMethod + +/** + * output for FindClassMode + */ +class FindClassModeHtmlWriter( + private val secResults: OutputSecResults, + val rule: IRule, + private val callMap: Set, + private val vulnerabilityMethod: SootMethod +) : HtmlWriter(rule.desc), AddVulnerabilityAndSaveResult { + override fun genContent(tag: TagConsumer<*>) { + tag.div { + genVulInfo(this.consumer) + genVulnerabilityPosition(this.consumer) + genInstance(this.consumer) + } + } + + + private fun genVulnerabilityPosition(tag: TagConsumer<*>) { + tag.a { + classes = setOf(classVulnerabilityDetail) + +"vulnerability position" + } + genMethodJimple(tag, vulnerabilityMethod) + genMethodJavaSource(tag, vulnerabilityMethod) + } + + private fun genInstance(tag: TagConsumer<*>) { + tag.a { + classes = setOf(classVulnerabilityDetail) + +"instance position" + } + for (site in callMap) { + genMethodWithHighlight(tag, site.method, setOf(site.stmt)) + genMethodJavaSource(tag, site.method) + } + } + + /** + * 1. generate HTML file + * 2. add this vulnerability to the final report + */ + override suspend fun addVulnerabilityAndSaveResultToOutput() { + val stringList: MutableList = ArrayList() + stringList.add(vulnerabilityMethod.signature) + val instantList: MutableList = ArrayList() + for (site in callMap) { + instantList.add(site.method.signature + "{ " + site.stmt.toString() + " }") + } + val tosUrl = saveContent(generateHtml(), htmlName) + secResults.addOneVulnerability( + VulnerabilityItem( + rule, + tosUrl, + FindClassModeVulnerability(stringList, vulnerabilityMethod.signature, instantList) + ) + ) + } +} + +class FindClassModeVulnerability( + override val target: List, + override val position: String, + private val instanceLocation: List +) : + IVulnerability { + override fun toDetail(): Map { + return mapOf( + "target" to target, + "InstanceLocation" to instanceLocation, + "position" to position + ) + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/ui/HtmlWriter.kt b/src/main/kotlin/net/bytedance/security/app/ui/HtmlWriter.kt new file mode 100644 index 0000000..303376a --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/ui/HtmlWriter.kt @@ -0,0 +1,249 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.ui + +import kotlinx.html.* +import kotlinx.html.stream.createHTML +import net.bytedance.security.app.EngineInfo +import net.bytedance.security.app.Log +import net.bytedance.security.app.RuleDescription +import net.bytedance.security.app.android.AndroidUtils +import net.bytedance.security.app.getConfig +import net.bytedance.security.app.web.DefaultVulnerabilitySaver +import soot.SootMethod +import soot.jimple.Stmt +import java.nio.charset.StandardCharsets +import java.time.LocalDateTime + +interface AddVulnerabilityAndSaveResult { + /** + * 1. generate HTML file + * 2. add this vulnerability to the final report + */ + suspend fun addVulnerabilityAndSaveResultToOutput() +} + +/** +html string generator for vulnerability report + */ +open class HtmlWriter(val desc: RuleDescription) { + val htmlName = desc.name + ".html" + fun generateHtml(): String { + return try { + createHTML(prettyPrint = false).html { + genHead(this.consumer) + genBody(this.consumer) + } + } catch (ex: Exception) { + ex.printStackTrace() + "" + } + } + + private fun genBody(tag: TagConsumer<*>) { + tag.body { + genContent(this.consumer) + } + } + + /** + * Generate basic information + */ + fun genVulInfo(tag: TagConsumer<*>) { + tag.a { + classes = setOf(classVulnerabilityDetail) + +"vulnerability detail" + } + tag.pre { + classes = setOf("java") + code { + +"Name: ${AndroidUtils.AppLabelName}\n" + +"PackageName: ${AndroidUtils.PackageName}\n" + +"ApplicationName: ${AndroidUtils.ApplicationName}\n" + +"VersionName: ${AndroidUtils.VersionName}\n" + +"VersionCode: ${AndroidUtils.VersionCode}\n" + +"MinSdk: ${AndroidUtils.MinSdk}\n" + +"TargetSdk: ${AndroidUtils.TargetSdk}\n" + desc.wiki?.let { + +String.format("wiki: ") + a { + href = it + target = "_blank" + +it + } + +"\n" + } + +"name: ${desc.name}\n" + +"category: ${desc.category}\n" + +"detail: ${desc.detail}\n" + if (getConfig().deobfApk.isNotEmpty()) { + +"deobfApk:" + a { + href = getConfig().deobfApk + target = "_blank" + +getConfig().deobfApk + } + +"\n" + } + desc.possibility?.let { + +"possibility: $it\n" + } + desc.model?.let { + +"model: $it\n" + } + desc.complianceCategory?.let { + +"complianceCategory: $it\n" + } + desc.complianceCategoryDetail?.let { + +"complianceCategoryDetail: $it\n" + } + +"scanTime: ${LocalDateTime.now()}\n" + +"engineVersion:${EngineInfo.Version}\n" + } + } + } + + /** + * need the concrete vulnerability to implement this method + */ + open fun genContent(tag: TagConsumer<*>) { + } + + companion object { + const val classVulnerabilityDetail = "vulnerability-detail" + private const val classCode = "code" + const val classJava = "java" + const val classBgheader1 = "bgheader1" + const val classBgheader2 = "bgheader2" + const val classHighlight = "highlightcode" + fun genHead(tag: TagConsumer<*>) { + tag.head { + meta { + httpEquiv = "Content-Type" + content = "text/html" + charset = "UTF-8" + } + + title("vulnerability scan result") + link { + rel = "stylesheet" + href = "https://cdnjs.cloudflare.com/ajax/libs/highlight.js/10.5.0/styles/default.min.css" + } + style { + unsafe { + raw( + """ + .$classBgheader1{ + background:#FFF; + color:#000; + } + .$classBgheader2{ + background:#FFF; + color:#00F; + } + .background { + background-color: #272727; + color: #ccc; + } + .$classCode { + background-color: #272727; + color: #ccc; + } + .$classVulnerabilityDetail{ + color:#F00; + } + .$classHighlight { + background:#FF0; + color:#00F; + } + + """.trimIndent() + ) + } + } + script { src = "https://cdnjs.cloudflare.com/ajax/libs/highlight.js/10.5.0/highlight.min.js" } + script { unsafe { raw("hljs.initHighlightingOnLoad();") } } + } + } + + /** + * generate the jimple body of method + */ + fun genMethodJimple(tag: TagConsumer<*>, method: SootMethod) { + genMethodWithHighlight(tag, method, HashSet()) + } + + /** + * generate the jimple body of method with highlight stmt + */ + fun genMethodWithHighlight(tag: TagConsumer<*>, method: SootMethod, hightSet: Set) { + tag.pre { + code { + classes = setOf("java") + +"${method.signature}{\n" + var i = 1 + for (unit in method.activeBody.units) { + if (hightSet.contains(unit)) { + div { + classes = setOf(classHighlight) + +"$i: $unit" + } + } else { + +"$i: $unit\n" + } + i += 1 + } + +"}\n" + } + } + } + + + /** + * Generate Java source code for method, if it exists + */ + fun genMethodJavaSource(tag: TagConsumer<*>, method: SootMethod) { + val javaSourceCode = getJavaSource(method) + if (javaSourceCode != null) { + tag.div { + a { + classes = setOf(classVulnerabilityDetail) + +"java source code:" + } + tag.pre { + code { + classes = setOf("java") + +"class ${method.declaringClass.name}{\n" + +javaSourceCode + +"}\n" + } + } + } + } + } + } + + suspend fun saveContent(content: String, name: String): String { + val tosUrl = + DefaultVulnerabilitySaver.getVulnerabilitySaver() + .saveVulnerability(content.toByteArray(StandardCharsets.UTF_8), name) + Log.logDebug("htmlwriter write vulnerability to $tosUrl") + return tosUrl + } +} + diff --git a/src/main/kotlin/net/bytedance/security/app/ui/TaintPathModeHtmlWriter.kt b/src/main/kotlin/net/bytedance/security/app/ui/TaintPathModeHtmlWriter.kt new file mode 100644 index 0000000..7e8f439 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/ui/TaintPathModeHtmlWriter.kt @@ -0,0 +1,389 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.ui + + +import kotlinx.html.* +import net.bytedance.security.app.Log +import net.bytedance.security.app.MethodFinder +import net.bytedance.security.app.android.ComponentDescription +import net.bytedance.security.app.pathfinder.PathResult +import net.bytedance.security.app.pathfinder.TaintEdge +import net.bytedance.security.app.pathfinder.TaintFlowEdgeFinder +import net.bytedance.security.app.pointer.PLLocalPointer +import net.bytedance.security.app.pointer.PLPointer +import net.bytedance.security.app.result.IVulnerability +import net.bytedance.security.app.result.OutputSecResults +import net.bytedance.security.app.result.VulnerabilityItem +import net.bytedance.security.app.rules.DirectModeRule +import net.bytedance.security.app.rules.TaintFlowRule +import net.bytedance.security.app.taintflow.TaintAnalyzer +import soot.SootMethod +import soot.jimple.Stmt +import soot.jimple.internal.JIfStmt + +/** + * generate html content for the taint flow vulnerability + * @param result the path from source to sink + * @param rule the rule of this vulnerability + * @param analyzer contains the source and sink info + */ +class TaintPathModeHtmlWriter( + private val secResult: OutputSecResults, + val analyzer: TaintAnalyzer, + val result: PathResult, + val rule: TaintFlowRule, +) : HtmlWriter(rule.desc), AddVulnerabilityAndSaveResult { + private val ruleThroughAPISet: HashSet = HashSet() + + // functions that appear sequentially from the source to sink path that is the `result` + private val methodArr: MutableList = ArrayList() + + // It has the same length as methodArr, and the element corresponds to stmts used in methodArr + private val stmtsInMethod: MutableList> = ArrayList() + + // It has the same length as methodArr + private val edgesInMethod: MutableList> = ArrayList() + + + private val apiSearchStmtSet: MutableSet = HashSet() + private var sourceStmtSet: MutableSet = HashSet() + private var sinkStmtSet: MutableSet = HashSet() + + init { + if (rule is DirectModeRule && rule.throughAPI != null) { + val throughAPI = rule.throughAPI + throughAPI.MethodSignature?.let { methodSignatures -> + methodSignatures.forEach { methodSignature -> + val methods = MethodFinder.checkAndParseMethodSig(methodSignature) + ruleThroughAPISet.addAll(methods.map { it.signature }) + } + } + throughAPI.MethodName?.let { + ruleThroughAPISet.addAll(it) + } + } + + } + + override fun genContent(tag: TagConsumer<*>) { + genVulInfo(tag) + + tag.div { + div { + classes = setOf(classVulnerabilityDetail) + +"data flow:" + } + tag.pre { + tag.code { + classes = setOf(classJava) + for (ptr in result.curPath) { + +"${ptr}\n" + } + } + } + + div { + classes = setOf(classVulnerabilityDetail) + +"call stack: " + } + tag.pre { + tag.code { + classes = setOf(classJava) + for (sm in methodArr) { + +"${sm.signature}\n" + } + } + } + + genCodeDetail(this.consumer) + } + } + + private fun genCodeDetail(tag: TagConsumer<*>) { + tag.div { + classes = setOf(classVulnerabilityDetail) + +"code detail: " + } + tag.div { + for ((i, sm) in methodArr.withIndex()) { + val edgeList = edgesInMethod[i] + var edge: PLPointer + + pre { + code { + classes = setOf(classJava) + for (j in edgeList.indices) { + edge = edgeList[j] + div { + if (j % 2 == 1) { + classes = setOf(classBgheader1) + } else { + classes = setOf(classBgheader2) + } + +edge.toString() + } + } + } + } + pre { + code { + classes = setOf(classJava) + val stmtList = stmtsInMethod[i] + val chain = sm.activeBody.units + val totalLines = chain.size + val start = 1 + var end = totalLines + var index = 1 + for (stmt in chain) { + if (stmt == stmtList[stmtList.size - 1] || + sourceStmtSet.contains(stmt) || + sinkStmtSet.contains(stmt) + ) { + if (totalLines > 8) { + end = if (index > 4) { + index + 4 + } else { + 8 + } + } + } + index++ + } + index = 1 + var labelIndex = 1 + val gotoTgtLabelMap: MutableMap = HashMap() + for (unit in chain) { + val stmt = unit as Stmt + if (index >= start && index <= end) { + var labelName = "" + if (stmt is JIfStmt) { + val targetStmt = stmt.target + if (gotoTgtLabelMap.containsKey(targetStmt)) { + labelName = gotoTgtLabelMap[targetStmt]!! + } else { + labelName = "LABEL" + labelIndex++ + gotoTgtLabelMap[targetStmt] = labelName + } + } + if (sourceStmtSet.contains(stmt)) { + apiSearchStmtSet.add(stmt) + if (gotoTgtLabelMap.containsKey(stmt)) { + +"${gotoTgtLabelMap[stmt]}:" + } + div { + classes = setOf(classHighlight) + +"$index:->[Source] $stmt" + } + } else if (sinkStmtSet.contains(stmt)) { + apiSearchStmtSet.add(stmt) + if (gotoTgtLabelMap.containsKey(stmt)) { + +"${gotoTgtLabelMap[stmt]}:" + } + div { + classes = setOf(classHighlight) + +"$index:->[Sink] $stmt" + } + } else if (stmtList.contains(stmt)) { + apiSearchStmtSet.add(stmt) + val taintIndex = stmtList.indexOf(stmt) + if (gotoTgtLabelMap.containsKey(stmt)) { + +"${gotoTgtLabelMap[stmt]}:" + } + div { + classes = setOf(classHighlight) + +"$index:->[${taintIndex + 1}] $stmt" + } + } else { + if (stmt is JIfStmt) { + val condition = stmt.condition + +"$index: if $condition goto $labelName\n" + } else { + if (gotoTgtLabelMap.containsKey(stmt)) { + +"${gotoTgtLabelMap[stmt]}:\n" + } + +"$index: $stmt\n" + } + } + } + index++ + } + } + } + genMethodJavaSource(this.consumer, sm) + } + } + } + + + override suspend fun addVulnerabilityAndSaveResultToOutput() { + + val sourcePtr = result.curPath.first() + analyzer.data.ptrStmtMapSrcSink[sourcePtr]?.let { + sourceStmtSet = it + } + + val sinkPtr = result.curPath.last() + analyzer.data.ptrStmtMapSrcSink[sinkPtr]?.let { + sinkStmtSet = it + } + + + mergeTaintPath(methodArr, stmtsInMethod, edgesInMethod, result.curPath) + + val tosUrl = saveContent(generateHtml(), htmlName) + Log.logDebug("Write vulnerability taint mode to $tosUrl") + + val throughAPISet: MutableSet = HashSet() + for (throughStmt in apiSearchStmtSet) { + if (throughStmt.containsInvokeExpr()) { + val invokeExpr = throughStmt.invokeExpr + if (isMatchThroughAPI( + invokeExpr.methodRef.signature, + invokeExpr.methodRef.name + ) + ) { + throughAPISet.add(invokeExpr.methodRef.signature) + } + } + } + secResult.addOneVulnerability( + VulnerabilityItem( + rule, tosUrl, + TaintPathModeVulnerability( + result.curPath.map { it.signature() }, + (sourcePtr as PLLocalPointer).method.signature, + sourcePtr, sinkPtr, + throughAPISet, analyzer.entryMethod + ) + ) + ) + } + + fun isMatchThroughAPI(s1: String, s2: String): Boolean { + return ruleThroughAPISet.contains(s1) || ruleThroughAPISet.contains(s2) + } + + companion object { + class Range(var start: Int, var end: Int) { + override fun toString(): String { + return "$start-$end" + } + } + + /** + calculate each statement corresponding to the propagation path + */ + fun getTaintEdges(curPath: List): List> { + val result = ArrayList>() + assert(curPath.isNotEmpty()) + var lastRange = Range(0, 0) + for (i in 0 until curPath.size - 1) { + val cur = curPath[i] + val next = curPath[i + 1] + val edges = TaintFlowEdgeFinder.getPossibleEdge(cur, next) + if (edges != null) { + if (lastRange.end > 0) { + lastRange.end = i //the first step is @data + } + for (edge in edges) { + lastRange = Range(lastRange.end, i + 1) + result.add(Pair(edge, lastRange)) + } + } + } + lastRange.end = curPath.size - 1 + return result + } + + fun mergeTaintPath( + methodArr: MutableList, + stmtsInMethod: MutableList>, + edgesInMethod: MutableList>, + curPath: List + ) { + var stmts: MutableList = ArrayList() + var edges: MutableList = ArrayList() + val path = ArrayList(curPath) + val taintEdges = getTaintEdges(path) + var prevMethod: SootMethod? = null + for (edge in taintEdges) { + if (edge.first.method != prevMethod && prevMethod != null) { + methodArr.add(prevMethod) + stmtsInMethod.add(stmts) + edgesInMethod.add(edges) + stmts = ArrayList() + edges = ArrayList() + } + prevMethod = edge.first.method + stmts.add(edge.first.stmt) + var start = edge.second.start + if (edges.size > 0 && edges.last() == path[start]) { + start++ + } + edges.addAll(path.subList(start, edge.second.end + 1)) + } + if (stmts.isNotEmpty()) { + methodArr.add(prevMethod!!) + stmtsInMethod.add(stmts) + edgesInMethod.add(edges) + } + } + + } +} + +class TaintPathModeVulnerability( + override val target: List, + override val position: String, + val source: PLPointer, + val sink: PLPointer, + val throughAPI: Set, + val entryMethod: SootMethod, +) : + IVulnerability { + private var manifest: ComponentDescription? = null + private val otherComponents: ArrayList = ArrayList() + override fun toDetail(): Map { + val m: MutableMap = mutableMapOf( + "target" to target.map { it }, + "position" to position, + "Source" to listOf(source.toString()), + "Sink" to listOf(sink.toString()), + "entryMethod" to entryMethod.toString() + ) + if (manifest != null) { + m["Manifest"] = manifest!! + } + if (otherComponents.isNotEmpty()) { + m["OtherComponents"] = otherComponents + } + if (throughAPI.isNotEmpty()) { + m["throughAPI"] = throughAPI + } + return m + } + + fun addManifest(com: ComponentDescription) { + if (manifest != null) { + otherComponents.add(com) + } else { + manifest = com + } + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/ui/util.kt b/src/main/kotlin/net/bytedance/security/app/ui/util.kt new file mode 100644 index 0000000..dfa4d83 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/ui/util.kt @@ -0,0 +1,147 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.ui + +import net.bytedance.security.app.Log +import net.bytedance.security.app.PLUtils +import net.bytedance.security.app.android.AndroidUtils +import net.bytedance.security.app.engineconfig.isLibraryClass +import net.bytedance.security.app.util.JavaAST +import soot.Scene +import soot.SootMethod +import soot.options.Options +import java.io.* +import java.nio.charset.StandardCharsets +import java.nio.file.Files + +/** +return method's full Java source code, + */ +internal fun getJavaSource(method: SootMethod): String? { + if (AndroidUtils.dexToJavaProcess == null) { + return null + } + val methodSig = method.signature +// val anonymousMethodSig: String? = null +// val declaringClassName = method.declaringClass.name +// if (declaringClassName.contains("$")) { +//// val sootMethodSetMap = ctx.findInstantCallSite(declaringClassName) +//// if (sootMethodSetMap.size == 1) { +//// anonymousMethodSig = +//// methodSig.substring(0, methodSig.indexOf('$')) + methodSig.substring(methodSig.indexOf(':')) +//// methodSig = ctx.findInstantCallSite(sm.declaringClass.name).iterator().next().method.signature +//// } else { +//// methodSig = methodSig.substring(0, methodSig.indexOf('$')) + methodSig.substring(methodSig.indexOf(':')) +//// } +// } + genJavaSourceCache(methodSig) + val methodBody = JavaAST.ASTMap[methodSig] ?: return null +// var anonymousMethodBody = "" +// if (anonymousMethodSig != null) { +// genJavaSourceCache(anonymousMethodSig) +// anonymousMethodBody = JavaAST.ASTMap[anonymousMethodSig] ?: "" +// } + return formatJavaCode(methodBody) +} + +fun formatJavaCode(methodBody: String): String { + val br = BufferedReader( + InputStreamReader( + ByteArrayInputStream( + (methodBody).toByteArray( + StandardCharsets.UTF_8 + ) + ), + StandardCharsets.UTF_8 + ) + ) + val out = StringBuilder() + var line: String + var space = 0 + try { + while (true) { + line = br.readLine() ?: break + out.append(" ") + if (line.endsWith("}")) { + space = line.length - 1 + out.append(line) + out.append("\n") + } else if (line.startsWith(" catch")) { + for (i in 0 until space) { + out.append(" ") + } + out.append(line) + out.append("\n") + } else if (line.startsWith(" else")) { + for (i in 0 until space) { + out.append(" ") + } + val newline = if (line.contains("if (")) { + val regex = Regex("if \\(") + val arr = line.split(regex).toTypedArray() + " else if(${arr[1]}" + } else { + line + } + out.append(newline) + } else { + out.append(line) + out.append("\n") + } + } + } catch (e: IOException) { + e.printStackTrace() + } + return out.toString() +} + +private fun genJavaSourceCache(methodSig: String) { + val sootMethod = Scene.v().grabMethod(methodSig) ?: return + val sc = sootMethod.declaringClass + try { + if (isLibraryClass(sc.name)) { + return + } + val javaSource = loadClass(sc.name) + if (javaSource != null) { + JavaAST.parseJavaSource(sc.name, javaSource) + } + } catch (e: Throwable) { + e.printStackTrace() + Log.logErr("ERROR java source load " + sc.name) + } +} + +/** + * Loads the source code for the specified class + */ +private fun loadClass(className: String): String? { + val javaSrcPath = + Options.v().output_dir() + PLUtils.JAVA_SRC + "app/src/main/java/" + className.replace(".", "/") + ".java" + val javaFile = File(javaSrcPath) + if (!javaFile.exists()) { +// PLLog.logErr("ERROR java file not exist $javaSrcPath") + return null + } + try { + return String(Files.readAllBytes(javaFile.toPath())) + } catch (e: Exception) { + Log.logErr("ERROR java file read error $javaSrcPath") + } + return null +} diff --git a/src/main/kotlin/net/bytedance/security/app/util/JavaAST.kt b/src/main/kotlin/net/bytedance/security/app/util/JavaAST.kt new file mode 100644 index 0000000..d68254a --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/util/JavaAST.kt @@ -0,0 +1,95 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +@file:Suppress("unused") + +package net.bytedance.security.app.util + +import org.eclipse.jdt.core.dom.* +import java.util.concurrent.ConcurrentHashMap + +/** + * generate java source for method + */ +object JavaAST { + /** + * key is a method, value is this method's source code + */ + var ASTMap: MutableMap = ConcurrentHashMap() + + private fun createMethodSig(className: String, node: MethodDeclaration) { + val nameArr = className.split("\\.".toRegex()).toTypedArray() + val lastName = nameArr[nameArr.size - 1] + var signature = "<" + className.replace("$", ".") + ": " + signature += if (node.returnType2 == null) { + "void " + } else { + node.returnType2.toString() + " " + } + var isInit = false + var signatureInit = signature + if (lastName == node.name.toString()) { + signatureInit += "" + isInit = true + } else { + signatureInit += node.name + } + signature += node.name + signature += "(" + signatureInit += "(" + if (node.parameters().size > 0) { + var i = 0 + while (i < node.parameters().size - 1) { + val param = node.parameters()[i] as SingleVariableDeclaration + signature += param.type.toString().replace("$", ".") + "," + signatureInit += param.type.toString().replace("$", ".") + "," + i++ + } + val param = node.parameters()[i] as SingleVariableDeclaration + signature += param.type.toString().replace("$", ".") + signatureInit += param.type.toString().replace("$", ".") + } + signature += ")" + signatureInit += ")" + signature += ">" + signatureInit += ">" + ASTMap[signature] = node.toString() + if (isInit) { + ASTMap[signatureInit] = node.toString() + } + } + + fun parseJavaSource(javaSource: String) { + val parser = ASTParser.newParser(AST.JLS15) + parser.setSource(javaSource.toCharArray()) + parser.setKind(ASTParser.K_COMPILATION_UNIT) + parser.createAST(null) as CompilationUnit + } + + fun parseJavaSource(className: String, javaSource: String) { + val parser = ASTParser.newParser(AST.JLS15) + parser.setSource(javaSource.toCharArray()) + parser.setKind(ASTParser.K_COMPILATION_UNIT) + val cu = parser.createAST(null) as CompilationUnit + cu.accept(object : ASTVisitor() { + override fun visit(node: MethodDeclaration): Boolean { + createMethodSig(className, node) + return true + } + }) + } +} diff --git a/src/main/kotlin/net/bytedance/security/app/util/Json.kt b/src/main/kotlin/net/bytedance/security/app/util/Json.kt new file mode 100644 index 0000000..241957c --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/util/Json.kt @@ -0,0 +1,74 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.util + +import at.syntaxerror.json5.Json5Module +import kotlinx.serialization.SerializationStrategy +import kotlinx.serialization.decodeFromString +import kotlinx.serialization.encodeToString +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.JsonElement +import kotlinx.serialization.json.decodeFromJsonElement +import kotlinx.serialization.json.encodeToJsonElement + +/** + * A simple wrapper of Json, supports JSON5 + */ +object Json { + var format = Json { + ignoreUnknownKeys = true + prettyPrint = true + } + + /** + * Json5 is parsed and converted to valid JSON + */ + val j5 = Json5Module { + allowInfinity = true + } + + inline fun decodeFromString(string: String): T { + val s2 = j5.encodeToString(j5.decodeObject(string)) + return format.decodeFromString(s2) + } + + fun parseToJsonElement(string: String): JsonElement { + val s2 = j5.encodeToString(j5.decodeObject(string)) + return format.parseToJsonElement(s2) + } + + inline fun decodeFromJsonElement(json: JsonElement): T { + return format.decodeFromJsonElement(json) + } + + inline fun encodeToString(value: T): String { + return format.encodeToString(value) + } + + inline fun encodeToPrettyString(value: T): String { + return format.encodeToString(value) + } + + inline fun encodeToJsonElement(value: T): JsonElement { + return format.encodeToJsonElement(value) + } + + inline fun encodeToJsonElement(serializer: SerializationStrategy, value: T): JsonElement { + return format.encodeToJsonElement(serializer, value) + } +} diff --git a/src/main/kotlin/net/bytedance/security/app/util/Profiler.kt b/src/main/kotlin/net/bytedance/security/app/util/Profiler.kt new file mode 100644 index 0000000..3504ccd --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/util/Profiler.kt @@ -0,0 +1,407 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +@file:Suppress("PropertyName", "unused", "OPT_IN_IS_NOT_ENABLED") + +package net.bytedance.security.app.util + + +import kotlinx.coroutines.* +import kotlinx.serialization.KSerializer +import kotlinx.serialization.Serializable +import kotlinx.serialization.Transient +import kotlinx.serialization.descriptors.PrimitiveKind +import kotlinx.serialization.descriptors.PrimitiveSerialDescriptor +import kotlinx.serialization.descriptors.SerialDescriptor +import kotlinx.serialization.encoding.Decoder +import kotlinx.serialization.encoding.Encoder +import kotlinx.serialization.json.* +import kotlinx.serialization.serializer +import net.bytedance.security.app.Log.logErr +import net.bytedance.security.app.Log.logInfo +import net.bytedance.security.app.PreAnalyzeContext +import net.bytedance.security.app.result.Results +import net.bytedance.security.app.result.model.AppInfo +import net.bytedance.security.app.taintflow.AnalyzeContext +import net.bytedance.security.app.taintflow.TaintAnalyzer +import net.bytedance.security.app.web.DefaultVulnerabilitySaver +import java.lang.management.ManagementFactory +import java.lang.management.MemoryUsage +import java.text.SimpleDateFormat +import java.util.* +import java.util.concurrent.atomic.AtomicBoolean + + +@Serializable(with = TimeRangeSerializer::class) +open class TimeRange(var startTime: Long = 0) { + var takes: Long = -1 + fun start() { + startTime = System.currentTimeMillis() + } + + fun end() { + this.takes = System.currentTimeMillis() - startTime + } +} + + +object TimeRangeSerializer : KSerializer { + override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("TimeRange", PrimitiveKind.STRING) + + override fun serialize(encoder: Encoder, value: TimeRange) { + val element = buildJsonObject { + if (value.startTime > 0) { + val date = Date(value.startTime) + // format of the date + // format of the date + val jdf = SimpleDateFormat("yyyy-MM-dd HH:mm:ss S") + val javaDate = jdf.format(date) + + put("startTime", Json.encodeToJsonElement(javaDate)) + } + if (value.takes >= 0) { + put("takes", Json.encodeToJsonElement(value.takes)) + } + } + encoder.encodeSerializableValue(serializer(), element) + } + + override fun deserialize(decoder: Decoder): TimeRange { + require(decoder is JsonDecoder) // this class can be decoded only by Json + val element = decoder.decodeJsonElement() + val m = element.jsonObject.toMutableMap() + val timeRange = TimeRange() + if (m.containsKey("startTime")) { + val date = m["startTime"]?.jsonPrimitive?.content ?: "" + val jdf = SimpleDateFormat("yyyy-MM-dd HH:mm:ss S") + val javaDate = jdf.parse(date) + timeRange.startTime = javaDate.time + } + if (m.containsKey("takes")) { + timeRange.takes = m["takes"]?.jsonPrimitive?.long ?: -1 + } + return timeRange + } +} + + +@Serializable +class ProcessMethodStatistics { + /* + getAvailMethodsAndClass + */ + var availableMethods = 0 + var availableClasses = 0 + + var DirectCallGraph = 0 + var HeirCallGraph = 0 + var DirectReverseCallGraph = 0 + var HeirReverseCallGraph = 0 +} + +var profiler = Profiler() + + +@Serializable +class Profiler { + var ApkFile = "" + var AppInfo: AppInfo? = null + var ProcessMethodStatistics = ProcessMethodStatistics() + + var totalRange = TimeRange(System.currentTimeMillis()) + val fragments: TimeRange = TimeRange() + + fun init() { + totalRange.start() + } + + fun initProcessMethodStatistics(methods: Int, classes: Int, ctx: PreAnalyzeContext) { + this.ProcessMethodStatistics.apply { + this.availableMethods = methods + this.availableClasses = classes + + this.DirectCallGraph = ctx.callGraph.directCallGraph.size + this.HeirCallGraph = ctx.callGraph.heirCallGraph.size + this.DirectReverseCallGraph = ctx.callGraph.directReverseCallGraph.size + this.HeirReverseCallGraph = ctx.callGraph.heirReverseCallGraph.size + } + } + + + private fun mapConvert(m: MutableMap>): Map> { + val m2 = HashMap>() + for ((k, v) in m) { + val l = ArrayList() + for (s in v) { + l.add(s) + } + m2[k] = ArrayList(l.sorted()) + } + return LinkedHashMap(m2.toSortedMap()) + } + + var stage = "" + + + suspend fun finishAndSaveProfilerResult(stage: String = ""): String { + totalRange.end() + this.stage = stage + val s = this.toString() + val tosUrl = DefaultVulnerabilitySaver.getVulnerabilitySaver() + .saveVulnerability(s.toByteArray(Charsets.UTF_8), "profiler.json") + logErr("stage=$stage Write profiler json to $tosUrl") + return tosUrl + } + + override fun toString(): String { + return Json.encodeToPrettyString(this) + } + + var parseApk: TimeRange = TimeRange() + + var preProcessor: TimeRange = TimeRange() + + var ruleAnalyzerCount: MutableMap = HashMap() + + @Synchronized + fun setRuleAnalyzerCount(rule: String, n: Int) { + val n1 = ruleAnalyzerCount[rule] + if (n1 == null) { + ruleAnalyzerCount[rule] = n + } else { + ruleAnalyzerCount[rule] = n1 + n + } + } + + + var uploadTosTakes = 0L + + @Synchronized + fun addUploadTosTakes(takes: Long) { + uploadTosTakes += takes + } + + var checkAndParseMethodSigInternalUse = 0L + + @Synchronized + fun checkAndParseMethodSigInternalTake(n: Long) { + this.checkAndParseMethodSigInternalUse += n + } + + var taintAnalyzerCount = 0 + var twoStagePointerAnalyzeCount = 0 + + @Synchronized + fun addTaintAnalyzerCount() { + this.taintAnalyzerCount++ + } + + @Synchronized + fun addTwoStagePointerAnalyzeCount() { + this.twoStagePointerAnalyzeCount++ + } + + var ptrLocalCount = 0 + var ptrStaticFieldCount = 0 + var ptrObjectFieldCount = 0 + var objectCount = 0 + + @Synchronized + fun newPtrLocal(@Suppress("UNUSED_PARAMETER") s: String) { + this.ptrLocalCount += 1 + } + + @Synchronized + fun newPtrObjectField(@Suppress("UNUSED_PARAMETER") s: String) { + this.ptrObjectFieldCount += 1 + } + + fun newPtrStaticField(@Suppress("UNUSED_PARAMETER") s: String) { + this.ptrStaticFieldCount += 1 + } + + fun newObject(@Suppress("UNUSED_PARAMETER") s: String) { + this.objectCount += 1 + } + + + val pointAnalyze: MutableMap = HashMap() + val taintPathCalc: MutableMap = HashMap() + + @Synchronized + fun startPointAnalyze(name: String) { + val tr = TimeRange() + tr.start() + pointAnalyze[name] = tr + } + + @Synchronized + fun stopPointAnalyze(name: String) { + pointAnalyze[name]!!.end() + } + + @Serializable + data class AnalyzeContextData( + val reachableMethods: Int, + val pointerToObjectSet: Int, + val pointerFlowGraph: Int, + val variableFlowGraph: Int, + val objects: Int, + val pointers: Int + ) + + val AnalyzeContextMap = HashMap() + + @Synchronized + fun entryContext(key: String, ctx: AnalyzeContext) { + AnalyzeContextMap[key] = AnalyzeContextData( + ctx.rm.size, + ctx.pointerToObjectSet.size, + ctx.pointerFlowGraph.size, + ctx.variableFlowGraph.size, + ctx.pt.objIndexMap.size, + ctx.pt.ptrIndexMap.size + ) + } + + @Synchronized + fun startTaintPathCalc(name: String) { + val tr = TimeRange() + tr.start() + taintPathCalc[name] = tr + } + + @Synchronized + fun stopTaintPathCalc(name: String) { + taintPathCalc[name]!!.end() + } + + + private var vulnerabilitiesCount = Vulnerability() + fun processResult(r: Results) { + var n = 0 + r.SecurityInfo.forEach { securityItems -> + for ((name, item) in securityItems.value) { + item.vulnerabilityItemMutableList.let { + n += it.size + vulnerabilitiesCount.categoryMap[name] = it.size + } + } + } + r.ComplianceInfo.forEach { + for ((name, item) in it.value) { + n += item.vulnerabilityItemMutableList.size + vulnerabilitiesCount.categoryMap[name] = item.vulnerabilityItemMutableList.size + } + } + this.vulnerabilitiesCount.total = n + } + + suspend fun startProfilerTaskInternal(isStopped: AtomicBoolean) { + var count = 0 + while (!isStopped.get()) { + try { + var s = 0 + while (s < 600 && !isStopped.get()) { + delay(1000) + s += 1 + } + profiler.finishAndSaveProfilerResult("ProfilerTask count=$count") + count += 1 + } catch (ex: Exception) { + ex.printStackTrace() + } + } + } + + @Transient + private val isStopped = AtomicBoolean(false) + + @OptIn(DelicateCoroutinesApi::class) + private fun startProfilerTask() { + GlobalScope.launch(CoroutineName("profilerTask")) { + startProfilerTaskInternal(isStopped) + } + } + + private fun stopProfilerTask() { + isStopped.set(true) + } + + var maxMemoryUsage: Long = 0 + + @Transient + val memoryTaskQuit = AtomicBoolean(false) + fun startMemoryProfile() { + startProfilerTask() + @OptIn(DelicateCoroutinesApi::class) + GlobalScope.launch(CoroutineName("memoryProfile")) { + while (!memoryTaskQuit.get()) { + val bean = ManagementFactory.getMemoryMXBean() + val memoryUsage: MemoryUsage = bean.heapMemoryUsage + if (maxMemoryUsage < memoryUsage.used) { + maxMemoryUsage = memoryUsage.used + } + logInfo("memory usage=${memoryUsage}") + delay(1000) + } + } + } + + fun stopMemoryProfile() { + memoryTaskQuit.set(true) + stopProfilerTask() + } + + @Serializable + data class TaintAnalyzerItem( + val entry: String, + val rule: String, + val sourceSize: Int, + val sinkSize: Int, + val depth: Int, + val sources: List, + val sinks: List + ) + + val analyzers = ArrayList() + fun setAnalyzers(arg: List) { + arg.forEach { analyzer -> + var sources = analyzer.sourcePtrSet.map { it.signature() }.toList() + var sinks = analyzer.sinkPtrSet.map { it.signature() }.toList() + if (sources.size > 3) { + sources = sources.slice(0..2) + } + if (sinks.size > 3) { + sinks = sinks.slice(0..2) + } + analyzers.add( + TaintAnalyzerItem( + analyzer.entryMethod.signature, + analyzer.rule.name, + analyzer.sourcePtrSet.size, + analyzer.sinkPtrSet.size, + analyzer.thisDepth, + sources, sinks + ) + ) + } + } +} + +@Serializable +data class Vulnerability(var total: Int = 0, val categoryMap: MutableMap = HashMap()) diff --git a/src/main/kotlin/net/bytedance/security/app/util/TaskQueue.kt b/src/main/kotlin/net/bytedance/security/app/util/TaskQueue.kt new file mode 100644 index 0000000..47f27d6 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/util/TaskQueue.kt @@ -0,0 +1,96 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.util + +import kotlinx.coroutines.* +import kotlinx.coroutines.channels.Channel +import net.bytedance.security.app.Log +import java.util.* + +/** + * A simple multithreaded task wrapper + */ +class TaskQueue( + private val name: String, + private val numberThreads: Int, + private val action: suspend (TaskData, Int) -> Unit +) { + private val queue = Channel(numberThreads * 2) + + /** + * Add a task + */ + suspend fun addTask(taskData: TaskData, isLast: Boolean = false) { + queue.send(taskData) + if (isLast) { + queue.close() + } + } + + /** + * all tasks are added + */ + fun addTaskFinished() { + queue.close() + } + + /** + * Be sure to run this function before addTask + */ + suspend fun runTask(): Job { + val scope = CoroutineScope(Dispatchers.Default) + val jobs = ArrayList() + for (i in 0 until numberThreads) { + val job = scope.launch(CoroutineName("$name-$i")) { + for (taskData in queue) { + action(taskData, i) + } + } + jobs.add(job) + } + return scope.launch(CoroutineName("$name-joinAll")) { jobs.joinAll() } + } +} + + +suspend fun runInMilliSeconds(job: Job, milliSeconds: Long, name: String, timeoutAction: () -> Unit) { + val start = System.currentTimeMillis() + val timer = Timer() + timer.schedule(object : TimerTask() { + override fun run() { + runBlocking { + if (job.isActive) { + Log.logWarn("$name runInMilliSeconds timeout") + val cancelStart = System.currentTimeMillis() + job.cancelAndJoin() + val cancelEnd = System.currentTimeMillis() + if (cancelEnd - cancelStart > 1000) { + Log.logWarn("$name runInMilliSeconds cancelAndJoin takes ${cancelEnd - cancelStart}") + } + timeoutAction() + } + } + } + }, milliSeconds) + job.join() + timer.cancel() + val end = System.currentTimeMillis() + if ((end - start - milliSeconds).toDouble() / milliSeconds > 0.1) { + Log.logWarn("$name runInMilliSeconds cost more than expected expect=$milliSeconds, actual=${end - start}") + } +} diff --git a/src/main/kotlin/net/bytedance/security/app/util/dataClassToString.kt b/src/main/kotlin/net/bytedance/security/app/util/dataClassToString.kt new file mode 100644 index 0000000..6f84ac1 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/util/dataClassToString.kt @@ -0,0 +1,41 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.util + +import net.bytedance.security.app.Log +import kotlin.reflect.full.memberProperties + +@Suppress("unused") +fun dataClassToString(instance: Any) { + val sb = StringBuilder() + sb.append("data class ${instance::class.qualifiedName} (") + var prefix = "" + instance::class.memberProperties.forEach { + sb.append(prefix) + prefix = "," + val call = try { + it.getter.call(instance) + } catch (ex: Exception) { + "" + } + + sb.append("${it.name} = $call") + } + sb.append(")") + Log.logDebug(sb.toString()) +} diff --git a/src/main/kotlin/net/bytedance/security/app/util/debug.kt b/src/main/kotlin/net/bytedance/security/app/util/debug.kt new file mode 100644 index 0000000..ce11ac5 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/util/debug.kt @@ -0,0 +1,53 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +@file:Suppress("unused") + +package net.bytedance.security.app.util + +import soot.SootMethod + +var dodebug = false +fun toSorted(s: Set): Set { + if (dodebug) { + return s.toSortedSet(compareBy { it.toString() }) + } + return s +} + +fun toSortedMap(s: Map): Map { + if (dodebug) { + return s.toSortedMap(compareBy { it.toString() }) + } + return s +} + + +fun toSortedMap2(s: Map): Map { + if (dodebug) { + return s.toSortedMap(compareBy { it.signature }) + } + return s +} + + +fun toSortedList(s: List): List { + if (dodebug) { + s.sortedWith(compareBy { it.toString() }) + } + return s +} diff --git a/src/main/kotlin/net/bytedance/security/app/util/helper.kt b/src/main/kotlin/net/bytedance/security/app/util/helper.kt new file mode 100644 index 0000000..6945773 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/util/helper.kt @@ -0,0 +1,185 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +@file:Suppress("unused") + +package net.bytedance.security.app.util + +import net.bytedance.security.app.Log +import net.bytedance.security.app.Log.logErr +import net.bytedance.security.app.web.DefaultVulnerabilitySaver +import java.util.* +import kotlin.system.exitProcess + +/** + * soot method signature parser + */ +class FunctionSignature( + var className: String, + var returnType: String, + var functionName: String, + var args: MutableList, +) + +fun FunctionSignature.subSignature(): String { + var s = "${this.returnType} ${this.functionName}(" + s += args.joinToString(separator = ",") + ")" + return s +} + +private enum class ParseState { + Class, ReturnType, Argument, FunctionName +} + +/** + * @param methodSig for example: + * @return FunctionSignature + */ +fun methodSignatureDestruction(methodSig: String): FunctionSignature { + if (!(methodSig.startsWith("<") && methodSig.endsWith(">") && methodSig.contains(": "))) { + Log.logFatal("Format Error $methodSig") + } + var s = "" + val fd = FunctionSignature("", "", "", ArrayList()) + var state = ParseState.Class + // " + for (i in 1 until methodSig.length - 1) { + when (val c = methodSig[i]) { + ':' -> if (state != ParseState.Class) { + exitProcess(-2) + } else { + // state = ParseState.Space + } + ' ' -> { + when (state) { + ParseState.Class -> { + fd.className = s + state = ParseState.ReturnType + s = "" + } + ParseState.ReturnType -> { + fd.returnType = s + s = "" + state = ParseState.FunctionName + } + else -> exitProcess(-7) + } + } + ',' -> { + when (state) { + ParseState.Argument -> { + fd.args.add(s) + s = "" + } + else -> exitProcess(-8) + } + } + '(' -> { + when (state) { + ParseState.FunctionName -> { + fd.functionName = s + s = "" + state = ParseState.Argument + } + else -> exitProcess(-9) + } + } + ')' -> { + when (state) { + ParseState.Argument -> { + fd.args.add(s) + s = "" + } + else -> exitProcess(-10) + } + } + else -> s += c + } + } + + return fd +} + +/** + * retrieves a function signature from a string, returning the string itself if none is found + * @param s for example: ref=()>, + * @return for example: ()> + */ +fun getMethodSigFromStr(s: String): String { + if (s.contains("<") && s.contains(">")) { + val start = s.indexOf('<') + val end = s.lastIndexOf('>') + if (start < end) { + return s.substring(start, end + 1) + } + } + return s +} + +suspend fun uploadJsonResult(outHtmlName: String, jsonBuf: String) { + val tosUrl = DefaultVulnerabilitySaver.getVulnerabilitySaver() + .saveVulnerability(jsonBuf.toByteArray(Charsets.UTF_8), outHtmlName) + logErr("Write All Vulnerabilities to $tosUrl") +} + +fun Map.toSortedMap(): SortedMap { + return this.toSortedMap(compareBy { it.toString() }) +} + +fun Set.toSortedSet(): SortedSet { + return this.toSortedSet(compareBy { it.toString() }) +} + +fun SortedSet.toFormatedString(): String { + var s = "" + for (k in this) { + s += "$k\n" + } + return s +} + +fun SortedMap.toFormatedString(): String { + var s = "" + for ((k, v) in this) { + s += "$k=$v\n" + } + return s +} + +/** + * String is a string like p0,p1,p2,... + * @return 0,1,2 + */ +fun String.argIndex(): Int { + return replace("p", "").toInt() +} + +/** + *is this string a valid jimple function signature + * for example: ()> + */ +fun String.isMethodSignature(): Boolean { + return this.startsWith("<") && this.endsWith(">") && this.contains("(") +} + +/** + * is this string a valid field signature + * for example: + */ +fun String.isFieldSignature(): Boolean { + return this.startsWith("<") && this.endsWith(">") && !this.contains("(") +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/web/VulnerabilityFileSaver.kt b/src/main/kotlin/net/bytedance/security/app/web/VulnerabilityFileSaver.kt new file mode 100644 index 0000000..6475d24 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/web/VulnerabilityFileSaver.kt @@ -0,0 +1,52 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.web + +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.withContext +import net.bytedance.security.app.getConfig +import java.io.File +import java.util.concurrent.atomic.AtomicInteger + +class VulnerabilityFileSaver : VulnerabilitySaver { + val path: String + private val count = AtomicInteger(0) + + init { + val base = getConfig().outPath + val vulnerabilityDirectory = "$base/vulnerability" + val vulnerabilityFile = File(vulnerabilityDirectory) + if (!vulnerabilityFile.exists()) { + vulnerabilityFile.mkdirs() + } + path = vulnerabilityDirectory + } + + override suspend fun saveVulnerability(outBuf: ByteArray, fileName2: String): String = withContext(Dispatchers.IO) { + val n = count.getAndAdd(1) + val fileName = "$path/$n-$fileName2" + val f = File(fileName) + try { + f.writeBytes(outBuf) + return@withContext f.absolutePath + } catch (ex: Exception) { + ex.printStackTrace() + return@withContext "" + } + } +} \ No newline at end of file diff --git a/src/main/kotlin/net/bytedance/security/app/web/VulnerabilitySaver.kt b/src/main/kotlin/net/bytedance/security/app/web/VulnerabilitySaver.kt new file mode 100644 index 0000000..d6e17d4 --- /dev/null +++ b/src/main/kotlin/net/bytedance/security/app/web/VulnerabilitySaver.kt @@ -0,0 +1,34 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.web + +interface VulnerabilitySaver { + suspend fun saveVulnerability(outBuf: ByteArray, fileName2: String): String +} + +object DefaultVulnerabilitySaver { + private var vulnerabilitySaver: VulnerabilitySaver = VulnerabilityFileSaver() + fun getVulnerabilitySaver(): VulnerabilitySaver { + return vulnerabilitySaver + } + + @Suppress("unused") + fun setVulnerabilitySaver(vulnerabilitySaver: VulnerabilitySaver) { + this.vulnerabilitySaver = vulnerabilitySaver + } +} \ No newline at end of file diff --git a/src/test/kotlin/at/syntaxerror/json5/DecodeJson5ObjectTest.kt b/src/test/kotlin/at/syntaxerror/json5/DecodeJson5ObjectTest.kt new file mode 100644 index 0000000..78fc0c6 --- /dev/null +++ b/src/test/kotlin/at/syntaxerror/json5/DecodeJson5ObjectTest.kt @@ -0,0 +1,78 @@ +package at.syntaxerror.json5 + +import kotlinx.serialization.json.JsonObject +import org.junit.jupiter.api.Test +import java.io.IOException +import java.nio.file.Files +import java.nio.file.Paths + +class DecodeJson5ObjectTest { + @Test + fun testDeocodeAndEncodeToJson() { +// create and configure the Json5Module + val j5 = Json5Module { + allowInfinity = true + indentFactor = 4u + } + + val json5 = """ + { + // comments + unquoted: 'and you can quote me on that', + singleQuotes: 'I can use "double quotes" here', + lineBreaks: "Look, Mom! \ + No \\n's!", + hexadecimal: 0xdecaf, + leadingDecimalPoint: .8675309, + andTrailing: 8675309., + positiveSign: +1, + trailingComma: 'in objects', + andIn: [ + 'arrays', + ], + "backwardsCompatible": "with JSON", + } + """.trimIndent() + +// Parse a JSON5 String to a Kotlinx Serialization JsonObject + val jsonObject: JsonObject = j5.decodeObject(json5) + +// encode the JsonObject to a Json5 String + val jsonString = j5.encodeToString(jsonObject) + + println(jsonString) +/* +{ + "unquoted": "and you can quote me on that", + "singleQuotes": "I can use \"double quotes\" here", + "lineBreaks": "Look, Mom! No \\n's!", + "hexadecimal": 912559, + "leadingDecimalPoint": 0.8675309, + "andTrailing": 8675309.0, + "positiveSign": 1, + "trailingComma": "in objects", + "andIn": [ + "arrays" + ], + "backwardsCompatible": "with JSON" +} +*/ + } + + @Test + fun testReadConfigFile() { + val jsonStr = try { + String(Files.readAllBytes(Paths.get("config/config.json5"))) + } catch (e: IOException) { + e.printStackTrace() + return + } + val j5 = Json5Module { + allowInfinity = true + } + val obj = j5.decodeObject(jsonStr) + val s = j5.encodeToString(obj) + println(s) + } + +} diff --git a/src/test/kotlin/net/bytedance/security/app/LogTest.kt b/src/test/kotlin/net/bytedance/security/app/LogTest.kt new file mode 100644 index 0000000..bcf63a8 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/LogTest.kt @@ -0,0 +1,38 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app + +import net.bytedance.security.app.Log.logDebug +import net.bytedance.security.app.Log.logErr +import net.bytedance.security.app.Log.logInfo +import org.junit.jupiter.api.Test + +internal class LogTest { + @Test + fun testLog() { + val nnn = "\n".repeat(10) + logErr(nnn) + logInfo("aaa") + logDebug("bbbb") + Thread.sleep(3000) + logErr("ccccc") + logInfo("dddd") + Thread.sleep(3000) + logErr("eeee") + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/MethodFinderTest.kt b/src/test/kotlin/net/bytedance/security/app/MethodFinderTest.kt new file mode 100644 index 0000000..37d6a65 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/MethodFinderTest.kt @@ -0,0 +1,121 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app + +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Test +import test.SootHelper +import test.TestHelper + +internal class MethodFinderTest { + init { + SootHelper.initSoot( + "MethodFinderTest", + listOf("${TestHelper.getTestClassSourceFileDirectory(this.javaClass.name)}/preprocess/testdata") + ) + } + + @Test + fun checkAndParseMethodSig() { + //1. Full function signature + var sig = "" + var methods = MethodFinder.checkAndParseMethodSig(sig) + + assertEquals(methods.map { it.signature }.toList(), listOf(sig)) + + //2. Base + sig = "" + methods = MethodFinder.checkAndParseMethodSig(sig) + + //Sub2 is not included because methodImplementedInSub is not implemented in Sub2 + assertEquals( + methods.map { it.signature }.toSortedSet().toList(), + listOf( + sig, + "" + ).toSortedSet().toList() + ) + + //3. method name is * + sig = "" + methods = MethodFinder.checkAndParseMethodSig(sig) + + assertEquals( + methods.map { it.signature }.toSortedSet().toList(), + listOf( + "", + "", + "", + "()>", + "", + "", + "", + "", + ).toSortedSet().toList() + ) + + //4. class name is * + sig = "<*: java.lang.Object methodImplementedInSub()>" + methods = MethodFinder.checkAndParseMethodSig(sig) + assertEquals( + methods.map { it.signature }.toSortedSet().toList(), + listOf( + "", + "", + "" + ) + ) + + //5. The return value is *, which is not an exact match + sig = "" + methods = MethodFinder.checkAndParseMethodSig(sig) + assertEquals( + methods.map { it.signature }.toSortedSet().toList(), listOf( + "", + "" + ).toSortedSet().toList() + ) + + //6. The parameter is *, which is not an exact match + sig = "" + methods = MethodFinder.checkAndParseMethodSig(sig) + assertEquals( + methods.map { it.signature }.toSortedSet().toList(), listOf( + "" + ).toSortedSet().toList() + ) + + //7. The function name partially matches + sig = "" + methods = MethodFinder.checkAndParseMethodSig(sig) + assertEquals( + methods.map { it.signature }.toSortedSet().toList(), listOf( + "", + "" + ).toSortedSet().toList() + ) + + //8. method that doesn't exist + sig = "" + methods = MethodFinder.checkAndParseMethodSig(sig) + assertEquals( + methods.map { it.signature }.toSortedSet().toList(), listOf() + ) + + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/PreAnalyzeContextTest.kt b/src/test/kotlin/net/bytedance/security/app/PreAnalyzeContextTest.kt new file mode 100644 index 0000000..cbeccf0 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/PreAnalyzeContextTest.kt @@ -0,0 +1,59 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app + +import kotlinx.coroutines.runBlocking +import net.bytedance.security.app.rules.IRulesForContext +import org.junit.jupiter.api.Test +import test.SootHelper +import test.TestHelper + +internal class PreAnalyzeContextTest() { + init { + SootHelper.initSoot( + "ContextTest", + listOf("${TestHelper.getTestClassSourceFileDirectory(this.javaClass.name)}/preprocess/testdata") + ) + } + + class Rules : IRulesForContext { + override fun constStringPatterns(): Set { + return setOf("field_const_str") + } + + override fun newInstances(): Set { + return setOf("android.webview.WebView") + } + + override fun fields(): Set { + return setOf() + } + + } + + @Test + fun testRun() { + val c = PreAnalyzeContext() + runBlocking { + c.createContext(Rules(), true) + } + assert(c.getMethodCounter() > 0) + assert(c.getClassCounter() > 0) + assert(c.constStringPatternMap.contains("field_const_str")) + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/RuleDataTest.kt b/src/test/kotlin/net/bytedance/security/app/RuleDataTest.kt new file mode 100644 index 0000000..474311e --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/RuleDataTest.kt @@ -0,0 +1,34 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app + + +import net.bytedance.security.app.ui.data +import net.bytedance.security.app.util.Json +import org.junit.jupiter.api.Test + +internal class RuleDataTest { + + @Test + fun decode() { + + val s = data + val rules: Map = Json.decodeFromString(s) + println(rules) + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/android/ComponentDescriptionDataSerializerTest.kt b/src/test/kotlin/net/bytedance/security/app/android/ComponentDescriptionDataSerializerTest.kt new file mode 100644 index 0000000..d081c12 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/android/ComponentDescriptionDataSerializerTest.kt @@ -0,0 +1,32 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.android + +import net.bytedance.security.app.util.Json +import org.junit.jupiter.api.Test + +internal class ComponentDescriptionDataSerializerTest { + @Test + fun testSerial() { + val c = ComponentDescription() + c.exported = true + c.otherMap["aaa"] = 37 + val s = Json.encodeToString(c) + println(s) + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/engineconfig/EngineConfigTest.kt b/src/test/kotlin/net/bytedance/security/app/engineconfig/EngineConfigTest.kt new file mode 100644 index 0000000..b696112 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/engineconfig/EngineConfigTest.kt @@ -0,0 +1,43 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.engineconfig + +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Assertions.assertTrue +import org.junit.jupiter.api.Test +import soot.Scene +import test.SootHelper + +internal class EngineConfigTest { + init { + soot.G.reset() + assertEquals(Scene.v().classes.size, 0) + } + + @Test + fun loadValidConfig() { + assertTrue(EngineConfig.libraryConfig.isLibraryClass("java.lang.String")) + } + + + @Test + fun visitCallbackAfterInitSoot() { + SootHelper.initSoot("visitCallbackAfterInitSoot", listOf()) + EngineConfig.callbackConfig.getCallBackConfig() + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/java/JavaASTTest.kt b/src/test/kotlin/net/bytedance/security/app/java/JavaASTTest.kt new file mode 100644 index 0000000..55a4940 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/java/JavaASTTest.kt @@ -0,0 +1,77 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.java + +import kotlinx.serialization.encodeToString +import kotlinx.serialization.json.Json +import net.bytedance.security.app.util.JavaAST +import org.junit.jupiter.api.Test + +val testdata = """ +package net.bytedance.security.app.pathfinder.testdata; + +public class CHATest { + static public class Base { + public Object getSource() { + return ""; + } + } + + static public class Sub extends Base { + @Override + public Object getSource() { + return Taint.source(); + } + } + + static public class ClassFlow { + void callsink(Object arg) { + Taint.sink(arg); + } + + Object f(Base b) { + return b.getSource(); + } + + void flow() { + Base b = new Base(); + Object obj = f(b); + callsink(obj); + } + } + +} +""".trimIndent() + +internal class JavaASTTest { + @Test + fun testParseJavaSource() { + try { + JavaAST.parseJavaSource("net.bytedance.security.app.pathfinder.testdata.CHATest", testdata) + assert(JavaAST.ASTMap.isNotEmpty()) + println(Json.encodeToString(JavaAST.ASTMap)) + } catch (ex: Exception) { + ex.printStackTrace() + assert(false) { + error("exception") + } + } + + } +} + diff --git a/src/test/kotlin/net/bytedance/security/app/model/SecurityVulnerabilityItemTest.kt b/src/test/kotlin/net/bytedance/security/app/model/SecurityVulnerabilityItemTest.kt new file mode 100644 index 0000000..313972c --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/model/SecurityVulnerabilityItemTest.kt @@ -0,0 +1,60 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.model + + +import net.bytedance.security.app.android.ComponentDescription +import net.bytedance.security.app.android.ToMapSerializeHelper +import net.bytedance.security.app.result.model.SecurityVulnerabilityItem +import net.bytedance.security.app.util.Json +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Test + +internal class SecurityVulnerabilityItemTest { + @Test + fun testSerial() { + val sv = SecurityVulnerabilityItem(details = HashMap(), hash = "aaa", possibility = "bbbb") + val m = HashMap() + m["target"] = "aaa" + val c = ComponentDescription() + c.exported = true + c.otherMap["fun"] = listOf("ccccc") + val c2: ToMapSerializeHelper = c + m["manifest"] = c2 + sv.details = m + val s = Json.encodeToString(sv) + assertEquals( + """ +{ + "details": { + "manifest": { + "exported": true, + "fun": [ + "ccccc" + ] + }, + "target": "aaa" + }, + "hash": "aaa", + "possibility": "bbbb" +} +""".trimIndent(), + s + ) + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/pathfinder/TaintFlowEdgeFinderTest.kt b/src/test/kotlin/net/bytedance/security/app/pathfinder/TaintFlowEdgeFinderTest.kt new file mode 100644 index 0000000..ea2f9c8 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/pathfinder/TaintFlowEdgeFinderTest.kt @@ -0,0 +1,316 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.pathfinder + +import kotlinx.coroutines.runBlocking +import net.bytedance.security.app.PLUtils +import net.bytedance.security.app.PreAnalyzeContext +import net.bytedance.security.app.pointer.* +import net.bytedance.security.app.preprocess.AnalyzePreProcessor +import net.bytedance.security.app.preprocess.MethodFieldConstCacheVisitor +import net.bytedance.security.app.preprocess.MethodSSAVisitor +import net.bytedance.security.app.preprocess.MethodStmtFieldCache +import net.bytedance.security.app.taintflow.TwoStagePointerAnalyze +import net.bytedance.security.app.taintflow.TwoStagePointerAnalyzeTest.Companion.createDefaultTwoStagePointerAnalyze +import net.bytedance.security.app.ui.TaintPathModeHtmlWriter +import org.junit.jupiter.api.Assertions +import org.junit.jupiter.api.Test +import soot.Scene +import soot.SootMethod +import soot.UnknownType +import soot.jimple.Stmt +import test.SootHelper +import test.TestHelper +import java.util.* + +internal class TaintFlowEdgeFinderTest { + init { + val ctx = PreAnalyzeContext() + val cam = AnalyzePreProcessor(10, ctx) + + SootHelper.initSoot( + "TaintFlowEdgeFinderTest", + listOf("${TestHelper.getTestClassSourceFileDirectory(this.javaClass.name)}/testdata") + ) + cam.addMethodVisitor { + MethodSSAVisitor() + }.addMethodVisitor { + MethodFieldConstCacheVisitor( + ctx, + MethodStmtFieldCache(), + HashSet(), HashSet(), HashSet() + ) + } + runBlocking { cam.run() } + } + + fun getZipSlipContext(): TwoStagePointerAnalyze { + try { + val entry = Scene.v() + .getMethod("") + val tsp = createDefaultTwoStagePointerAnalyze(entry) + runBlocking { + tsp.doPointerAnalyze() + } + return tsp + } catch (ex: Exception) { + val path = System.getProperty("user.dir") + throw Exception("Failed to load ZipSlip, please check the path: $path") + } + + } + + class Path(val src: PLPointer, val dst: PLPointer, val edges: List?) { + fun isValid(): Boolean { + if (edges != null && edges.isNotEmpty()) { + return true + } + if (src is PLPtrObjectField && src.field == PLUtils.DATA_FIELD) { + return true + } + if (dst is PLPtrObjectField && dst.field == PLUtils.DATA_FIELD) { + return true + } + return false + } + + override fun toString(): String { + return "$src->$dst: edges=${edges} " + } + } + + fun assertPathValid(path: List) { + val edges = ArrayList() + for (i in 0 until path.size - 1) { + val edge = TaintFlowEdgeFinder.getPossibleEdge(path[i], path[i + 1]) + edges.add(Path(path[i], path[i + 1], edge)) + } + edges.forEach { + Assertions.assertTrue(it.isValid(), "path=${it}") + } + + val edgesWithRange = TaintPathModeHtmlWriter.getTaintEdges(path) + println("path len=${path.size}") + edgesWithRange.forEach { + println("${it.first.method.name}======> ${it.second}") +// assertTrue(it.isValid(), "path=${it}") + } + val methods = ArrayList() + val stmts = ArrayList>() + val edgesInMethod = ArrayList>() + TaintPathModeHtmlWriter.mergeTaintPath(methods, stmts, edgesInMethod, path) + methods.forEachIndexed { index, sootMethod -> + println("index=$index, method=${sootMethod.name}") + stmts[index].forEach { + println("$it") + } + Assertions.assertEquals(stmts[index].size, HashSet(stmts[index]).size) + edgesInMethod[index].forEach { + println("$it") + } + Assertions.assertEquals(edgesInMethod[index].size, HashSet(edgesInMethod[index]).size) + } + } + + @Test + fun testZipSlip() { + //this case depends on the version of the Java compiler, only JDK 11.0.12 tests passed and JDK 1.8 tests failed + PLUtils.dumpClass("net.bytedance.security.app.pathfinder.testdata.ZipSlip") + val tsp = getZipSlipContext() + val unzipFolder = Scene.v() + .getMethod("") + val srcName = "\$r3" + val sinkName = "\$r21" + val srcPtr = tsp.pt.allocLocal(unzipFolder, srcName, UnknownType.v()) + val sinkPtr = tsp.pt.allocLocal(unzipFolder, sinkName, UnknownType.v()) + + val path = + TaintPathFinder.bfsSearch(srcPtr, setOf(sinkPtr), tsp.ctx.variableFlowGraph, 256, "test") + println("path=$path") + Assertions.assertTrue(path != null) + Assertions.assertEquals(3, path!!.size) + Assertions.assertEquals(srcPtr.signature(), path.first().signature()) + Assertions.assertEquals(sinkPtr.signature(), path.last().signature()) + assertPathValid(path) + } + + @Test + fun testZipSlipFromConstString() { + PLUtils.dumpClass("net.bytedance.security.app.pathfinder.testdata.ZipSlip") + val tsp = getZipSlipContext() + val unzipFolder = Scene.v() + .getMethod("") + + val sinkName = "\$r21" + + val srcPtr = tsp.pt.allocLocal(tsp.entryMethod, PLUtils.constStrSig("path1"), UnknownType.v()) + val sinkPtr = tsp.pt.allocLocal(unzipFolder, sinkName, UnknownType.v()) + + + val path = + TaintPathFinder.bfsSearch(srcPtr, setOf(sinkPtr), tsp.ctx.variableFlowGraph, 256, "test") + println("path=$path") + Assertions.assertTrue(path != null) + Assertions.assertEquals(9, path!!.size) + Assertions.assertEquals(srcPtr.signature(), path.first().signature()) + Assertions.assertEquals(sinkPtr.signature(), path.last().signature()) + assertPathValid(path) + } + + @Test + fun pathInstanceMethod() { + PLUtils.dumpClass("net.bytedance.security.app.pathfinder.testdata.TaintExample") + PLUtils.dumpClass("net.bytedance.security.app.pathfinder.testdata.Taint") + val entry = Scene.v() + .getMethod("") + val pt = PointerFactory() + val tsp = createDefaultTwoStagePointerAnalyze(entry) + runBlocking { + tsp.doPointerAnalyze() + } + + val srcPtr = pt.allocLocal( + Scene.v() + .getMethod(""), + "\$r0", + UnknownType.v() + ) + val falseSinkPtr = pt.allocLocal(entry, "\$r0", UnknownType.v()) + val allTaint = tsp.ctx.collectPropagation(srcPtr, false) + Assertions.assertTrue(!allTaint.contains(falseSinkPtr)) + val sinkPtr = pt.allocLocal(entry, "\$r1", UnknownType.v()) + Assertions.assertTrue(tsp.ctx.collectPropagation(srcPtr).contains(sinkPtr)) + val path = + TaintPathFinder.bfsSearch(srcPtr, setOf(sinkPtr), tsp.ctx.variableFlowGraph, 256, "test") + Assertions.assertTrue(path != null) + Assertions.assertEquals(3, path!!.size) + Assertions.assertEquals(srcPtr.signature(), path.first().signature()) + Assertions.assertEquals(sinkPtr.signature(), path.last().signature()) + assertPathValid(path) + } + + @Test + fun testInstanceDispatchNotUseCHA() { + PLUtils.dumpClass("net.bytedance.security.app.pathfinder.testdata.CHATest\$ClassFlow") + PLUtils.dumpClass("net.bytedance.security.app.pathfinder.testdata.CHATest\$Base") + PLUtils.dumpClass("net.bytedance.security.app.pathfinder.testdata.CHATest\$Sub") + val entry = Scene.v() + .getMethod("") + val pt = PointerFactory() + val tsp = createDefaultTwoStagePointerAnalyze(entry) + runBlocking { + tsp.doPointerAnalyze() + } + + val srcPtr = pt.allocLocal( + Scene.v() + .getMethod(""), + "\$r0", + UnknownType.v() + ) + val sinkPtr = pt.allocLocal( + Scene.v() + .getMethod(""), + "r0", + UnknownType.v() + ) + Assertions.assertFalse(tsp.ctx.collectPropagation(srcPtr).contains(sinkPtr)) + + val path = + TaintPathFinder.bfsSearch(srcPtr, setOf(sinkPtr), tsp.ctx.variableFlowGraph, 256, "test") + Assertions.assertNull(path) + } + + @Test + fun testStmt() { + PLUtils.dumpClass("net.bytedance.security.app.pathfinder.testdata.ZipSlip") + val clz = Scene.v().getSootClass("net.bytedance.security.app.pathfinder.testdata.ZipSlip") + for (method in clz.methods) { + println("method: ${method.shortSignature()}") + for (unit in method.activeBody.units) { + val stmt = unit as Stmt + println("stmt=${stmt}") + for (valueBox in stmt.useAndDefBoxes) { + println("${valueBox.value} --->${valueBox.value.javaClass.name}") + } + println("----") + } + } + } + + @Test + fun testBFS() { + val src = newNode(1) + var prev = newNode(2) + val g = HashMap>() + var last = prev + g[src] = setOf(prev) + for (i in 3 until 10) { + last = newNode(i) + g[prev] = setOf(last) + prev = last + } + + var path = TaintPathFinder.bfsSearch(src, setOf(last), g, 10, "test") + assert(path!!.size == 9) + println(path) + path = TaintPathFinder.bfsSearch(src, setOf(last), g, 9, "test") + assert(path == null) + path = TaintPathFinder.bfsSearch(src, setOf(last), g, 3, "test") + assert(path == null) //because of maxLen path, one path for test is discarded,currentLen=9,maxLen=9 + } + + @Test + fun testBFSSameSourceAndSink() { + val src = newNode(1) + var prev = newNode(2) + val g = HashMap>() + var last: PLPointer + g[src] = setOf(prev) + for (i in 3 until 10) { + last = newNode(i) + g[prev] = setOf(last) + prev = last + } + + val path = TaintPathFinder.bfsSearch(src, setOf(src), g, 10, "test") + assert(path!!.size == 1) + + } + + fun newNode(name: Int): PLPointer { + return PLLocalPointer(TwoStagePointerAnalyze.getPseudoEntryMethod(), name.toString(), UnknownType.v()) + } + + @Test + fun testlinkedlist() { + val queue = LinkedList() + queue.addLast(1) + queue.addLast(2) + queue.addLast(3) + Assertions.assertEquals(listOf(1, 2, 3), queue.toList()) + val n = queue.pollFirst() + Assertions.assertEquals(1, n) + Assertions.assertEquals(listOf(2, 3), queue.toList()) + queue.pollFirst() + Assertions.assertEquals(listOf(3), queue.toList()) + queue.pollFirst() + Assertions.assertEquals(listOf(), queue.toList()) + } + +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/pathfinder/TaintPathFinderTest.kt b/src/test/kotlin/net/bytedance/security/app/pathfinder/TaintPathFinderTest.kt new file mode 100644 index 0000000..544390b --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/pathfinder/TaintPathFinderTest.kt @@ -0,0 +1,96 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.pathfinder + +import kotlinx.coroutines.runBlocking +import net.bytedance.security.app.result.OutputSecResults +import net.bytedance.security.app.ruleprocessor.DirectModeProcessor +import net.bytedance.security.app.ruleprocessor.RuleProcessorFactory +import net.bytedance.security.app.ruleprocessor.RuleProcessorFactoryTest +import net.bytedance.security.app.rules.RuleFactory +import net.bytedance.security.app.rules.Rules +import net.bytedance.security.app.rules.TaintFlowRule +import net.bytedance.security.app.taintflow.TwoStagePointerAnalyzeTest.Companion.createDefaultTwoStagePointerAnalyze +import org.junit.jupiter.api.Assertions +import org.junit.jupiter.api.Test +import test.SootHelper +import test.TestHelper + +internal class TaintPathFinderTest { + init { + SootHelper.initSoot( + "TaintPathFinderTest", + listOf("${TestHelper.getTestClassSourceFileDirectory(this.javaClass.name)}/testdata") + ) + } + + + fun createTwoStagePointerAnalyzerFromRule(ruleFileName: String): TaintPathFinder { + val rules = Rules( + listOf( + "${TestHelper.getTestClassSourceFileDirectory(this.javaClass.name)}/testdata/$ruleFileName" + ), RuleFactory() + ) + val finder: TaintPathFinder + runBlocking { + rules.loadRules() + val ctx = RuleProcessorFactoryTest.createContext(rules) + val rp = RuleProcessorFactory.create(ctx, rules.allRules[0].mode) + rp.process(rules.allRules[0]) + val dmp = (rp as DirectModeProcessor) + val analyzer = dmp.analyzers[0] + val tsp = createDefaultTwoStagePointerAnalyze(analyzer.entryMethod) + tsp.doPointerAnalyze() + finder = TaintPathFinder(ctx, tsp.ctx, rules.allRules.first() as TaintFlowRule, analyzer) + } + return finder + } + + @Test + fun analyze() { + OutputSecResults.testClearVulnerabilityItems() + val finder = createTwoStagePointerAnalyzerFromRule("unzipslip.json") + runBlocking { + finder.analyze() + } + Assertions.assertEquals(2, OutputSecResults.vulnerabilityItems().size) + runBlocking { + OutputSecResults.processResult(finder.ctx) + } + } + + @Test + fun constStringAnalyze() { + OutputSecResults.testClearVulnerabilityItems() + val finder = createTwoStagePointerAnalyzerFromRule("unzipslipFromConstString.json") + runBlocking { + finder.analyze() + } + Assertions.assertEquals(1, OutputSecResults.vulnerabilityItems().size) + } + + @Test + fun pathBetweenMethods() { + OutputSecResults.testClearVulnerabilityItems() + val finder = createTwoStagePointerAnalyzerFromRule("another_example.json") + runBlocking { + finder.analyze() + } + Assertions.assertEquals(1, OutputSecResults.vulnerabilityItems().size) + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/pathfinder/testdata/CHATest.java b/src/test/kotlin/net/bytedance/security/app/pathfinder/testdata/CHATest.java new file mode 100644 index 0000000..2d05cc0 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/pathfinder/testdata/CHATest.java @@ -0,0 +1,33 @@ +package net.bytedance.security.app.pathfinder.testdata; + +public class CHATest { + static public class Base { + public Object getSource() { + return ""; + } + } + + static public class Sub extends Base { + @Override + public Object getSource() { + return Taint.source(); + } + } + + static public class ClassFlow { + void callsink(Object arg) { + Taint.sink(arg); + } + + Object f(Base b) { + return b.getSource(); + } + + void flow() { + Base b = new Base(); + Object obj = f(b); + callsink(obj); + } + } + +} diff --git a/src/test/kotlin/net/bytedance/security/app/pathfinder/testdata/Taint.java b/src/test/kotlin/net/bytedance/security/app/pathfinder/testdata/Taint.java new file mode 100644 index 0000000..474af78 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/pathfinder/testdata/Taint.java @@ -0,0 +1,14 @@ +package net.bytedance.security.app.pathfinder.testdata; + +public class Taint { + public static Object source() { + return new Object(); + } + + public static void sink(Object object) { + } + + public static Object sanitize(Object object) { + return object; + } +} diff --git a/src/test/kotlin/net/bytedance/security/app/pathfinder/testdata/TaintExample.java b/src/test/kotlin/net/bytedance/security/app/pathfinder/testdata/TaintExample.java new file mode 100644 index 0000000..ed59230 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/pathfinder/testdata/TaintExample.java @@ -0,0 +1,27 @@ +package net.bytedance.security.app.pathfinder.testdata; + +public class TaintExample { + static void TaintCrossStaticMethod() { + Taint.sink(staticSource2()); + } + + static Object staticSource1() { + return Taint.source(); + } + + static Object staticSource2() { + return staticSource1(); + } + + void TaintCrossInstanceMethod() { + Taint.sink(instanceSource2()); + } + + Object instanceSource1() { + return Taint.source(); + } + + Object instanceSource2() { + return instanceSource1(); + } +} diff --git a/src/test/kotlin/net/bytedance/security/app/pathfinder/testdata/ZipSlip.java b/src/test/kotlin/net/bytedance/security/app/pathfinder/testdata/ZipSlip.java new file mode 100644 index 0000000..b758357 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/pathfinder/testdata/ZipSlip.java @@ -0,0 +1,87 @@ +package net.bytedance.security.app.pathfinder.testdata; + + +import java.io.File; +import java.io.FileInputStream; +import java.io.FileOutputStream; +import java.util.zip.ZipEntry; +import java.util.zip.ZipInputStream; + +public class ZipSlip { + private String instanceField = "aaa"; + private static String staticField = "staticField"; + + public void UnZipFolder(String zipFileString, String outPathString) throws Exception { + ZipInputStream inZip = new ZipInputStream(new FileInputStream(zipFileString)); + ZipEntry zipEntry; + String szName = ""; + while ((zipEntry = inZip.getNextEntry()) != null) { + szName = zipEntry.getName(); + if (zipEntry.isDirectory()) { + szName = szName.substring(0, szName.length() - 1); + File folder = new File(outPathString + File.separator + staticField + szName + "constString"); + folder.mkdirs(); + } else { + File file = new File(outPathString + File.separator + instanceField + szName); + file.createNewFile(); + FileOutputStream out = new FileOutputStream(file); + int len; + byte[] buffer = new byte[1024]; + while ((len = inZip.read(buffer)) != -1) { + out.write(buffer, 0, len); + out.flush(); + } + out.close(); + } + } + inZip.close(); + } + + public void UnZipFolderFix1(String zipFileString, String outPathString) throws Exception { + ZipInputStream inZip = new ZipInputStream(new FileInputStream(zipFileString)); + ZipEntry zipEntry; + String szName = ""; + while ((zipEntry = inZip.getNextEntry()) != null) { + szName = zipEntry.getName(); + if (szName.contains("..")) { + throw new SecurityException("unzip slip"); + } + + if (zipEntry.isDirectory()) { + szName = szName.substring(0, szName.length() - 1); + File folder = new File(outPathString + File.separator + szName); + folder.mkdirs(); + } else { + File file = new File(outPathString + File.separator + szName); + file.createNewFile(); + FileOutputStream out = new FileOutputStream(file); + int len; + byte[] buffer = new byte[1024]; + while ((len = inZip.read(buffer)) != -1) { + out.write(buffer, 0, len); + out.flush(); + } + out.close(); + } + } + inZip.close(); + } + + public void f() throws Exception { + UnZipFolder("path1", "outpath1"); + UnZipFolderFix1("path2", "outpath2"); + } + + public String returnStringConst() { + return "aaa"; + } + + public String returnString() { + return staticField; + } + + public void printString() { + System.out.println(returnString()); + } +} + diff --git a/src/test/kotlin/net/bytedance/security/app/pathfinder/testdata/another_example.json b/src/test/kotlin/net/bytedance/security/app/pathfinder/testdata/another_example.json new file mode 100644 index 0000000..5874474 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/pathfinder/testdata/another_example.json @@ -0,0 +1,31 @@ +{ + "unZipSlipSliceMode": { + "DirectMode": true, + "traceDepth": 8, + "desc": { + "name": "unZipSlip", + "category": "FileRisk", + "detail": "", + "wiki": "", + "possibility": "", + "model": "" + }, + "entry": { + "methods": [ + "" + ] + }, + "source": { + "Return": [ + "" + ] + }, + "sink": { + "": { + "TaintCheck": [ + "p0" + ] + } + } + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/pathfinder/testdata/unzipslip.json b/src/test/kotlin/net/bytedance/security/app/pathfinder/testdata/unzipslip.json new file mode 100644 index 0000000..332b6ce --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/pathfinder/testdata/unzipslip.json @@ -0,0 +1,31 @@ +{ + "unZipSlipSliceMode": { + "DirectMode": true, + "traceDepth": 8, + "desc": { + "name": "unZipSlip", + "category": "FileRisk", + "detail": "", + "wiki": "", + "possibility": "", + "model": "" + }, + "entry": { + "methods": [ + "" + ] + }, + "source": { + "Return": [ + "" + ] + }, + "sink": { + "(*)>": { + "TaintCheck": [ + "p*" + ] + } + } + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/pathfinder/testdata/unzipslipFromConstString.json b/src/test/kotlin/net/bytedance/security/app/pathfinder/testdata/unzipslipFromConstString.json new file mode 100644 index 0000000..521e4be --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/pathfinder/testdata/unzipslipFromConstString.json @@ -0,0 +1,31 @@ +{ + "unZipSlipSliceMode": { + "DirectMode": true, + "traceDepth": 8, + "desc": { + "name": "unZipSlip", + "category": "FileRisk", + "detail": "", + "wiki": "", + "possibility": "", + "model": "" + }, + "entry": { + "methods": [ + "" + ] + }, + "source": { + "ConstString": [ + "path1" + ] + }, + "sink": { + "(*)>": { + "TaintCheck": [ + "p*" + ] + } + } + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/AnalyzePreProcessorTest.kt b/src/test/kotlin/net/bytedance/security/app/preprocess/AnalyzePreProcessorTest.kt new file mode 100644 index 0000000..470b234 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/preprocess/AnalyzePreProcessorTest.kt @@ -0,0 +1,172 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.preprocess + +import kotlinx.coroutines.CoroutineScope +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.channels.Channel +import kotlinx.coroutines.launch +import kotlinx.coroutines.runBlocking +import net.bytedance.security.app.DEBUG +import net.bytedance.security.app.Log +import net.bytedance.security.app.PLUtils +import net.bytedance.security.app.PreAnalyzeContext +import net.bytedance.security.app.util.Json +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Assertions.assertTrue +import org.junit.jupiter.api.Test +import soot.Scene +import test.SootHelper +import test.TestHelper + +internal open class AnalyzePreProcessorTest { + private val ctx = PreAnalyzeContext() + private val cam = AnalyzePreProcessor(10, ctx) + + init { + Log.setLevel(DEBUG) + SootHelper.initSoot( + "ClassAndMethodHandlerTest", + listOf("${TestHelper.getTestClassSourceFileDirectory(this.javaClass.name)}/testdata") + ) + } + + @Test + fun runCounter() { + cam.addMethodVisitor { MethodCounter(ctx) } + cam.addClassVisitor { ClassCounter(ctx) } + runBlocking { cam.run() } + println("ClassCounter: ${ctx.getClassCounter()}, MethodCounter: ${ctx.getMethodCounter()}") + assert(ctx.getMethodCounter() > 0) + assert(ctx.getClassCounter() > 0) + } + + @Test + fun runPattern() { + cam.addMethodVisitor { + MethodSSAVisitor() + }.addMethodVisitor { + MethodFieldConstCacheVisitor( + ctx, + MethodStmtFieldCache(), + setOf("SubField1"), HashSet(), HashSet() + ) + } + runBlocking { cam.run() } + println("ctx.patternMap=${ctx.constStringPatternMap}") + assertEquals(1, ctx.constStringPatternMap.size) + } + + @Test + fun testCallGraph() { + cam.addMethodVisitor { + MethodSSAVisitor() + }.addMethodVisitor { + MethodFieldConstCacheVisitor( + ctx, + MethodStmtFieldCache(), + HashSet(), HashSet(), HashSet() + ) + } + runBlocking { cam.run() } + PLUtils.dumpClass("net.bytedance.security.app.preprocess.testdata.Sub") + val m = Scene.v() + .getMethod("") + val dms = ctx.callGraph.directCallGraph[m] + val hms = ctx.callGraph.heirCallGraph[m] + assertTrue(dms!!.size >= 3, "it depends on java version") + assertTrue(hms!!.size >= 3, "it depends on java version") + val m2 = + Scene.v() + .getMethod("") + val rdms = ctx.callGraph.directCallGraph[m2] + val rhms = ctx.callGraph.heirCallGraph[m2] + assertEquals(1, rdms!!.size) + assertEquals(3, rhms!!.size) + println("caller=${m.signature}:\n direct callees=${dms} \n,heir callees=${hms}\n") + println("callee=${m2.signature}:\n direct callers=${rdms} \n,heir callers=${rhms}\n") + } + + @Test + fun testCallGraph2() { + cam.addMethodVisitor { + MethodSSAVisitor() + }.addMethodVisitor { + MethodFieldConstCacheVisitor( + ctx, + MethodStmtFieldCache(), + HashSet(), HashSet(), HashSet() + ) + } + runBlocking { cam.run() } + val directs = + ctx.callGraph.directCallGraph.filter { it.key.signature.indexOf("(5) + var sum = 0 + val scope = CoroutineScope(Dispatchers.Default) + val job = scope.launch { + for (i in chan) { + println(i) + sum += i + } + } + val job2 = scope.launch { + for (i in 1..10) { + chan.send(i) + } + chan.close() + } + runBlocking { + job.join() + job2.join() + } + assert(sum == 55) + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/CallGraphTest.kt b/src/test/kotlin/net/bytedance/security/app/preprocess/CallGraphTest.kt new file mode 100644 index 0000000..e5ecce8 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/preprocess/CallGraphTest.kt @@ -0,0 +1,123 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.preprocess + +import kotlinx.coroutines.runBlocking +import net.bytedance.security.app.DEBUG +import net.bytedance.security.app.Log +import net.bytedance.security.app.PreAnalyzeContext +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Assertions.assertTrue +import org.junit.jupiter.api.Test +import soot.Scene +import test.SootHelper +import test.TestHelper + +internal class CallGraphTest { + private val ctx = PreAnalyzeContext() + private val cam = AnalyzePreProcessor(10, ctx) + + init { + Log.setLevel(DEBUG) + SootHelper.initSoot( + "CallGraphTest", + listOf("${TestHelper.getTestClassSourceFileDirectory(this.javaClass.name)}/testdata") + ) + cam.addMethodVisitor { + MethodSSAVisitor() + }.addMethodVisitor { + MethodFieldConstCacheVisitor( + ctx, + MethodStmtFieldCache(), + HashSet(), HashSet(), HashSet() + ) + } + runBlocking { cam.run() } + } + + @Test + fun testSimple() { + val src = Scene.v() + .getMethod("") + val sink = Scene.v() + .getMethod("") + val ss = CallGraph.SourceAndSinkCross(false, src, sink, 10, false, ctx.callGraph) + val result = ss.traceAndCross() + assertTrue(result != null) + assertEquals( + "", + result!!.entryMethod.signature + ) + assertEquals(6, result.depth) + } + + @Test + fun testLessDepth() { + val src = Scene.v() + .getMethod("") + val sink = Scene.v() + .getMethod("") + val ss = CallGraph.SourceAndSinkCross(false, src, sink, 3, false, ctx.callGraph) + val result = ss.traceAndCross() + assertTrue(result == null) + } + + @Test + fun testHeirNotFound() { + val src = Scene.v() + .getMethod("") + val sink = Scene.v() + .getMethod("") + val ss = CallGraph.SourceAndSinkCross(false, src, sink, 10, false, ctx.callGraph) + val result = ss.traceAndCross() + assertTrue(result == null) + } + + @Test + fun testHeir() { + val src = Scene.v() + .getMethod("") + val sink = Scene.v() + .getMethod("") + val ss = CallGraph.SourceAndSinkCross(true, src, sink, 10, false, ctx.callGraph) + val result = ss.traceAndCross() + assertTrue(result != null) + assertEquals( + "", + result!!.entryMethod.signature + ) + assertEquals(6, result.depth) + } + + @Test + fun testHeir2() { + val src = Scene.v() + .getMethod("") + val sink = Scene.v() + .getMethod("") + val ss = CallGraph.SourceAndSinkCross(true, src, sink, 10, false, ctx.callGraph) + val result = ss.traceAndCross() + assertTrue(result != null) + assertEquals( + "", + result!!.entryMethod.signature + ) + assertEquals(6, result.depth) + } + +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/MethodCallbackVisitorTest.kt b/src/test/kotlin/net/bytedance/security/app/preprocess/MethodCallbackVisitorTest.kt new file mode 100644 index 0000000..db850d9 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/preprocess/MethodCallbackVisitorTest.kt @@ -0,0 +1,62 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.preprocess + +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Test +import soot.Scene + +internal class MethodCallbackVisitorTest : AnalyzePreProcessorTest() { + + @Test + fun reflectionTest() { + val m = Scene.v().getMethod("") + + MethodSSAVisitor.jimpleSSAPreprocess(m) + val beforeSize = m.activeBody.units.size + MethodCallbackVisitor(false).visitMethod(m) + val afterSize = m.activeBody.units.size + println(m.activeBody) + assertEquals(beforeSize + 3, afterSize) + } + + @Test + fun reflectionTest2() { + val m = Scene.v().getMethod("") + + MethodSSAVisitor.jimpleSSAPreprocess(m) + val beforeSize = m.activeBody.units.size + MethodCallbackVisitor(false).visitMethod(m) + val afterSize = m.activeBody.units.size + assertEquals(beforeSize + 6, afterSize) + } + + @Test + fun testCLInit() { + + val m = Scene.v().getMethod("()>") + + MethodSSAVisitor.jimpleSSAPreprocess(m) + val beforeSize = m.activeBody.units.size + MethodCallbackVisitor(false).visitMethod(m) + val afterSize = m.activeBody.units.size + assertEquals(beforeSize + 1, afterSize) + println("after:") + println(m.activeBody) + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/MethodFieldConstCacheVisitorTest.kt b/src/test/kotlin/net/bytedance/security/app/preprocess/MethodFieldConstCacheVisitorTest.kt new file mode 100644 index 0000000..2ba5569 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/preprocess/MethodFieldConstCacheVisitorTest.kt @@ -0,0 +1,210 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.preprocess + +import kotlinx.coroutines.runBlocking +import net.bytedance.security.app.PLUtils +import net.bytedance.security.app.PreAnalyzeContext +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Assertions.assertTrue +import org.junit.jupiter.api.Test +import soot.Scene + +internal class MethodFieldConstCacheVisitorTest + : AnalyzePreProcessorTest() { + private val ctx = PreAnalyzeContext() + private val cache = MethodStmtFieldCache() + + init { + val cam = AnalyzePreProcessor(10, ctx) + cam.addMethodVisitor { + MethodSSAVisitor() + } + runBlocking { cam.run() } + } + + fun makeEmptyVisitor(): MethodFieldConstCacheVisitor { + return MethodFieldConstCacheVisitor(ctx, cache, HashSet(), HashSet(), HashSet()) + } + + @Test + fun testVisitMethodCallInterface() { + PLUtils.dumpClass("net.bytedance.security.app.preprocess.testdata.Sub") + val v = makeEmptyVisitor() + val m = Scene.v() + .getMethod("") + v.visitMethod(m) + assertEquals( + v.cache.methodDirectRefs.map { it.key.signature }.toSortedSet().toList(), listOf( + "" + ) + ) + assertEquals( + v.cache.methodHeirRefs.map { it.key.signature }.toSortedSet().toList(), listOf( + "", + "" + ) + ) + } + + @Test + fun testVisitMethodCallMethodImplementedInParent() { + PLUtils.dumpClass("net.bytedance.security.app.preprocess.testdata.Sub") + val v = makeEmptyVisitor() + val m = Scene.v() + .getMethod("") + + v.visitMethod(m) + assert(v.cache.storeFieldRefs.isEmpty()) + println(v.cache.methodHeirRefs) + assertTrue(v.cache.methodHeirRefs.isEmpty()) + assertTrue(v.cache.newInstanceRefs.isEmpty()) + assertTrue(v.cache.methodDirectRefs.size >= 3, "it depends on the java version") + assertTrue( + v.cache.methodDirectRefs.map { it.key.signature }.toSortedSet().contains( + "" + ) + ) + assertTrue(v.cache.loadFieldRefs.isEmpty()) + assertTrue(v.cache.storeFieldRefs.isEmpty()) + } + + @Test + fun testVisitMethodField() { + val v = makeEmptyVisitor() + val m = Scene.v().getMethod("()>") + v.visitMethod(m) + assertEquals( + v.cache.storeFieldRefs.map { it.key.signature }.toSortedSet().toList(), listOf() + ) + + val v2 = MethodFieldConstCacheVisitor( + ctx, + cache, + HashSet(), + setOf(""), + HashSet() + ) + v2.visitMethod(m) + assert(v.cache.loadFieldRefs.isEmpty()) + println(v.cache.methodDirectRefs) + assertTrue(v.cache.methodDirectRefs.isNotEmpty()) //call base. + assertTrue(v.cache.methodHeirRefs.isEmpty()) + assertTrue(v.cache.newInstanceRefs.isEmpty()) + assertEquals(1, v.cache.storeFieldRefs.size) + assertEquals( + v2.cache.storeFieldRefs.map { it.key.signature }.toSortedSet().toList(), + listOf("") + ) + } + + @Test + fun testVisitMethodStaticField() { + val v = MethodFieldConstCacheVisitor( + ctx, + cache, + HashSet(), + setOf(""), + HashSet() + ) + val m = Scene.v().getMethod("()>") + v.visitMethod(m) + assert(v.cache.loadFieldRefs.isEmpty()) + assertTrue(v.cache.methodDirectRefs.isEmpty()) + assertTrue(v.cache.methodHeirRefs.isEmpty()) + assertTrue(v.cache.newInstanceRefs.isEmpty()) + assertEquals(1, v.cache.storeFieldRefs.size) + assertEquals( + v.cache.storeFieldRefs.map { it.key.signature }.toSortedSet().toList(), + listOf("") + ) + } + + + @Test + fun testVisitMethodLoadField() { + val v = MethodFieldConstCacheVisitor( + ctx, + cache, + HashSet(), + setOf(""), + HashSet() + ) + val m = Scene.v() + .getMethod("") + v.visitMethod(m) + assert(v.cache.storeFieldRefs.isEmpty()) + assertTrue(v.cache.methodDirectRefs.isNotEmpty()) + assertTrue(v.cache.methodHeirRefs.isEmpty()) //Object.toString is ignored + assertTrue(v.cache.newInstanceRefs.isEmpty()) + assertEquals(v.cache.loadFieldRefs.size, 1) + assertEquals( + v.cache.loadFieldRefs.map { it.key.signature }.toSortedSet().toList(), + listOf("") + ) + } + + @Test + fun testVisitMethodNewInstance() { + val v = MethodFieldConstCacheVisitor( + ctx, + cache, + HashSet(), + setOf(), + setOf( + "net.bytedance.security.app.preprocess.testdata.Sub", + "net.bytedance.security.app.preprocess.testdata.Sub2" + ) + ) + val m = Scene.v().getMethod("") + v.visitMethod(m) + assert(v.cache.storeFieldRefs.isEmpty()) +// assertTrue(v.c.methodDirectRefs.()) +// println(v.c.methodHeirRefs) + assertTrue(v.cache.methodHeirRefs.isEmpty()) + assertTrue(v.cache.loadFieldRefs.isEmpty()) + assertEquals(2, v.cache.newInstanceRefs.size) + assertEquals( + v.cache.newInstanceRefs.map { it.key.className }.toSortedSet().toList(), + listOf( + "net.bytedance.security.app.preprocess.testdata.Sub", + "net.bytedance.security.app.preprocess.testdata.Sub2" + ) + ) + } + + @Test + fun testSpecialInvoke() { + val v = MethodFieldConstCacheVisitor( + ctx, + cache, + HashSet(), + setOf(), + setOf( + ) + ) + val m = Scene.v() + .getMethod("") + v.visitMethod(m) + assertTrue(v.cache.methodHeirRefs.isEmpty()) + assertEquals( + "", + v.cache.methodDirectRefs.keys.first().signature + ) + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/MethodSSAVisitorTest.kt b/src/test/kotlin/net/bytedance/security/app/preprocess/MethodSSAVisitorTest.kt new file mode 100644 index 0000000..dada7ee --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/preprocess/MethodSSAVisitorTest.kt @@ -0,0 +1,51 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.preprocess + +import org.junit.jupiter.api.Test + +import soot.Scene +import soot.jimple.Stmt + +internal class MethodSSAVisitorTest : AnalyzePreProcessorTest() { + + @Test + fun visitMethod() { + val m = Scene.v() + .getMethod("") + + val body = MethodSSAVisitor.jimpleSSAPreprocess(m) + val body2 = MethodSSAVisitor.jimpleSSAPreprocess(m) + assert(body != body2) + + } + + @Test + fun sourceLineNumber() { + val m = Scene.v() + .getMethod("") + + val body = MethodSSAVisitor.jimpleSSAPreprocess(m) + for (u in body.units) { + val stmt = u as Stmt + val lineNumber = stmt.getTag("LineNumberTag") + println("stmt:$stmt,lineNumber:$lineNumber") + } + println(body) + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/SootConcurrentErrorTest.kt b/src/test/kotlin/net/bytedance/security/app/preprocess/SootConcurrentErrorTest.kt new file mode 100644 index 0000000..d18ba47 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/preprocess/SootConcurrentErrorTest.kt @@ -0,0 +1,79 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.preprocess + +import kotlinx.coroutines.runBlocking +import net.bytedance.security.app.PreAnalyzeContext +import org.junit.jupiter.api.Test +import soot.Scene +import soot.jimple.Stmt +import test.SootHelper +import test.TestHelper +import kotlin.concurrent.thread + +class SootConcurrentErrorTest { + private val ctx = PreAnalyzeContext() + private val cam = AnalyzePreProcessor(10, ctx) + + init { + SootHelper.initSootForClasses( + "SootConcurrentError", + "${TestHelper.getTestClassSourceFileDirectory(this.javaClass.name)}/classes" + ) + cam.addMethodVisitor { + MethodSSAVisitor() + } + runBlocking { cam.run() } + } + + @Test + fun testResolveMethodConcurrentError() { + val invokes = arrayOf( + (Scene.v() + .getMethod("()>").activeBody.units.toArray()[1] as Stmt).invokeExpr, + ( + Scene.v() + .getMethod("()>").activeBody.units.toArray()[1] as Stmt + ).invokeExpr, + ( + Scene.v() + .getMethod("()>").activeBody.units.toArray()[1] as Stmt + ).invokeExpr, + ( + Scene.v() + .getMethod("()>").activeBody.units.toArray()[1] as Stmt + ).invokeExpr, + ( + Scene.v() + .getMethod("()>").activeBody.units.toArray()[1] as Stmt + ).invokeExpr, + ) + val len = invokes.size + for (i in 1..10) { + val i2 = i + thread(start = true) { + Thread.sleep(1000) + val m = invokes[i2 % len].method //resolve method may lead to concurrent error + println(m.signature) + println("i2=$i2.ref=${invokes[i2 % len].methodRef}") + } + } + + Thread.sleep(5000) + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/classes/net/bytedance/security/app/preprocess/testdata/SuperNotExist.class b/src/test/kotlin/net/bytedance/security/app/preprocess/classes/net/bytedance/security/app/preprocess/testdata/SuperNotExist.class new file mode 100644 index 0000000..70a14d9 Binary files /dev/null and b/src/test/kotlin/net/bytedance/security/app/preprocess/classes/net/bytedance/security/app/preprocess/testdata/SuperNotExist.class differ diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/classes/net/bytedance/security/app/preprocess/testdata/SuperNotExist2.class b/src/test/kotlin/net/bytedance/security/app/preprocess/classes/net/bytedance/security/app/preprocess/testdata/SuperNotExist2.class new file mode 100644 index 0000000..fedf722 Binary files /dev/null and b/src/test/kotlin/net/bytedance/security/app/preprocess/classes/net/bytedance/security/app/preprocess/testdata/SuperNotExist2.class differ diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/classes/net/bytedance/security/app/preprocess/testdata/SuperNotExist3.class b/src/test/kotlin/net/bytedance/security/app/preprocess/classes/net/bytedance/security/app/preprocess/testdata/SuperNotExist3.class new file mode 100644 index 0000000..df1f19b Binary files /dev/null and b/src/test/kotlin/net/bytedance/security/app/preprocess/classes/net/bytedance/security/app/preprocess/testdata/SuperNotExist3.class differ diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/classes/net/bytedance/security/app/preprocess/testdata/SuperNotExist4.class b/src/test/kotlin/net/bytedance/security/app/preprocess/classes/net/bytedance/security/app/preprocess/testdata/SuperNotExist4.class new file mode 100644 index 0000000..26da22f Binary files /dev/null and b/src/test/kotlin/net/bytedance/security/app/preprocess/classes/net/bytedance/security/app/preprocess/testdata/SuperNotExist4.class differ diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/classes/net/bytedance/security/app/preprocess/testdata/SuperNotExist5.class b/src/test/kotlin/net/bytedance/security/app/preprocess/classes/net/bytedance/security/app/preprocess/testdata/SuperNotExist5.class new file mode 100644 index 0000000..5b586bc Binary files /dev/null and b/src/test/kotlin/net/bytedance/security/app/preprocess/classes/net/bytedance/security/app/preprocess/testdata/SuperNotExist5.class differ diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/Apple.java b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/Apple.java new file mode 100644 index 0000000..45ad8ba --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/Apple.java @@ -0,0 +1,52 @@ +package net.bytedance.security.app.preprocess.testdata; + +import java.lang.reflect.Constructor; +import java.lang.reflect.InvocationTargetException; +import java.lang.reflect.Method; + +public class Apple { + + private int price; + + public int getPrice() { + return price; + } + + public void setPrice(int price) { + this.price = price; + } + + public void normalCall() { + Apple apple = new Apple(); + apple.setPrice(5); + System.out.println("Apple Price:" + apple.getPrice()); + } + + public void reflectionCall() throws ClassNotFoundException, NoSuchMethodException, InvocationTargetException, InstantiationException, IllegalAccessException { + Class clz = Class.forName("net.bytedance.security.app.preprocess.testdata.Apple"); + Method setPriceMethod = clz.getMethod("setPrice", int.class); + Constructor appleConstructor = clz.getConstructor(); + Object appleObj = appleConstructor.newInstance(); + setPriceMethod.invoke(appleObj, 14); + Method getPriceMethod = clz.getMethod("getPrice"); + System.out.println("Apple Price:" + getPriceMethod.invoke(appleObj)); + } + + public void reflectionCall2() throws ClassNotFoundException, NoSuchMethodException, InvocationTargetException, InstantiationException, IllegalAccessException { + Class clz = Class.forName("net.bytedance.security.app.preprocess.testdata.Apple"); + Method setPriceMethod = clz.getMethod("setPrice", int.class); + Constructor appleConstructor = clz.getConstructor(); + Object appleObj = appleConstructor.newInstance(); + setPriceMethod.invoke(appleObj, 14); + Method getPriceMethod = clz.getMethod("getPrice"); + System.out.println("Apple Price:" + getPriceMethod.invoke(appleObj)); + + Class clz2 = Class.forName("net.bytedance.security.app.preprocess.testdata.Apple"); + Method setPriceMethod2 = clz2.getMethod("setPrice", int.class); + Constructor appleConstructor2 = clz2.getConstructor(); + Object appleObj2 = appleConstructor2.newInstance(); + setPriceMethod2.invoke(appleObj, 14); + Method getPriceMethod2 = clz2.getMethod("getPrice"); + System.out.println("Apple Price:" + getPriceMethod2.invoke(appleObj)); + } +} diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/Base.java b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/Base.java new file mode 100644 index 0000000..4b338d5 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/Base.java @@ -0,0 +1,16 @@ +package net.bytedance.security.app.preprocess.testdata; + + +public class Base implements Interface { + public Object methodImplementedInSub() { + return null; + } + + public Object methodImplementedInSub2() { + return null; + } + + public Object allImplemented() { + return null; + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/Interface.java b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/Interface.java new file mode 100644 index 0000000..56a6971 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/Interface.java @@ -0,0 +1,7 @@ +package net.bytedance.security.app.preprocess.testdata; + +public interface Interface { + public Object methodImplementedInSub(); + + public Object methodImplementedInSub2(); +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/InterfaceNonExist.java b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/InterfaceNonExist.java new file mode 100644 index 0000000..2a49757 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/InterfaceNonExist.java @@ -0,0 +1,5 @@ +package net.bytedance.security.app.preprocess.testdata; + +public interface InterfaceNonExist { + public Object noImplementationMethod(); +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/NotExist.java b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/NotExist.java new file mode 100644 index 0000000..95dbda4 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/NotExist.java @@ -0,0 +1,4 @@ +package net.bytedance.security.app.preprocess.testdata; + +public class NotExist { +} diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/Sub.java b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/Sub.java new file mode 100644 index 0000000..cc39226 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/Sub.java @@ -0,0 +1,44 @@ +package net.bytedance.security.app.preprocess.testdata; + + +public class Sub extends Base { + public static String s = "static_field_const_str"; + public String SubField1 = "SubField1"; + + public Object methodImplementedInSub() { + String s1 = s; + s1 += s; + s1 += SubField1; + return s1; + } + + public Object callMethodImplementedInParent() { + Object o1 = this.methodImplementedInSub2(); + Object o2 = this.methodImplementedInSub2(); + return o1.toString() + o2.toString(); + } + + public Object callInterface(Interface i) { + return i.methodImplementedInSub(); + } + + public Object callInterfaceNoImplementation(InterfaceNonExist i) { + return i.noImplementationMethod(); + } + + public static void newInstance() { + Interface b = new Sub(); + Interface b1 = new Base(); + Interface b2 = new Sub2(); + Base b3 = new Sub(); + Base b4 = new Sub2(); + } + + public void callConstString() { + System.out.println("conststring"); + } + + public Object allImplemented() { + return super.allImplemented(); + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/Sub2.java b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/Sub2.java new file mode 100644 index 0000000..d976fb9 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/Sub2.java @@ -0,0 +1,21 @@ +package net.bytedance.security.app.preprocess.testdata; + +public class Sub2 extends Base { + public Object field1 = null; + + Sub2() { + field1 = "field_const_str"; + } + + public Object methodImplementedInSub2() { + return new Object(); + } + + public Object anotherf() { + return this.field1; + } + + public Object allImplemented() { + return anotherf(); + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/SuperNotExist.java b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/SuperNotExist.java new file mode 100644 index 0000000..fdca6f6 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/SuperNotExist.java @@ -0,0 +1,7 @@ +package net.bytedance.security.app.preprocess.testdata; + +public class SuperNotExist extends NotExist { + SuperNotExist() { + super(); + } +} diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/SuperNotExist2.java b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/SuperNotExist2.java new file mode 100644 index 0000000..5e8c6ec --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/SuperNotExist2.java @@ -0,0 +1,7 @@ +package net.bytedance.security.app.preprocess.testdata; + +public class SuperNotExist2 extends NotExist { + SuperNotExist2() { + super(); + } +} diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/SuperNotExist3.java b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/SuperNotExist3.java new file mode 100644 index 0000000..d7f867d --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/SuperNotExist3.java @@ -0,0 +1,7 @@ +package net.bytedance.security.app.preprocess.testdata; + +public class SuperNotExist3 extends NotExist { + SuperNotExist3() { + super(); + } +} diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/SuperNotExist4.java b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/SuperNotExist4.java new file mode 100644 index 0000000..e90010e --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/SuperNotExist4.java @@ -0,0 +1,7 @@ +package net.bytedance.security.app.preprocess.testdata; + +public class SuperNotExist4 extends NotExist { + SuperNotExist4() { + super(); + } +} diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/SuperNotExist5.java b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/SuperNotExist5.java new file mode 100644 index 0000000..c07500a --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/SuperNotExist5.java @@ -0,0 +1,7 @@ +package net.bytedance.security.app.preprocess.testdata; + +public class SuperNotExist5 extends NotExist { + SuperNotExist5() { + super(); + } +} diff --git a/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/TestCallgraph.java b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/TestCallgraph.java new file mode 100644 index 0000000..c3f2963 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/preprocess/testdata/TestCallgraph.java @@ -0,0 +1,45 @@ +package net.bytedance.security.app.preprocess.testdata; + +public class TestCallgraph { + void calldirect(Sub s) { + df1(s); + sinkDirect(); + } + + void df1(Sub s) { + df2(s); + } + + void df2(Sub s) { + df3(s); + } + + void df3(Sub s) { + s.methodImplementedInSub(); + } + + void sinkDirect() { + sink(); + } + + void sink() { + + } + + void callHeir(Base b) { + hf1(b); + sinkDirect(); + } + + void hf1(Base s) { + hf2(s); + } + + void hf2(Base s) { + hf3(s); + } + + void hf3(Base s) { + s.methodImplementedInSub2(); + } +} diff --git a/src/test/kotlin/net/bytedance/security/app/ruleprocessor/DirectModeProcessorTest.kt b/src/test/kotlin/net/bytedance/security/app/ruleprocessor/DirectModeProcessorTest.kt new file mode 100644 index 0000000..09e9947 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/ruleprocessor/DirectModeProcessorTest.kt @@ -0,0 +1,85 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.ruleprocessor + +import kotlinx.coroutines.runBlocking +import net.bytedance.security.app.MethodFinder +import net.bytedance.security.app.PLUtils +import net.bytedance.security.app.rules.RuleFactory +import net.bytedance.security.app.rules.Rules +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.BeforeEach +import org.junit.jupiter.api.Test +import test.SootHelper +import test.TestHelper + +internal class DirectModeProcessorTest { + init { + SootHelper.initSoot( + "DirectModeProcessorTest", + listOf("${TestHelper.getTestClassSourceFileDirectory(this.javaClass.name)}/testdata") + ) + } + + @BeforeEach + fun clearCache() { + MethodFinder.clearCache() + } + + @Test + fun createAnalyzers() { + val rules = Rules( + listOf( + "${TestHelper.getTestClassSourceFileDirectory(this.javaClass.name)}/testdata/unZipSlipDirectMode.json" + ), RuleFactory() + ) + + runBlocking { + rules.loadRules() + val ctx = RuleProcessorFactoryTest.createContext(rules) + PLUtils.dumpClass("net.bytedance.security.app.ruleprocessor.testdata.ZipSlip") + val rp = RuleProcessorFactory.create(ctx, rules.allRules[0].mode) + rp.process(rules.allRules[0]) + val analyzers = (rp as DirectModeProcessor).analyzers + assertEquals(1, analyzers.size) + analyzers.forEach { + assertEquals(3, it.data.sourcePointerSet.size) + assertEquals(3, it.data.sinkPointerSet.size) + } + analyzers.forEachIndexed { index, taintAnalyzer -> + println("$index:") + println(taintAnalyzer.dump()) + } + SliceModeProcessorTest.mustContains( + analyzers, + "->\$r3", + "UnZipFolder" + ) + SliceModeProcessorTest.mustContains( + analyzers, + "->\$r3", + "UnZipFolder" + ) + SliceModeProcessorTest.mustContains( + analyzers, + "->\$r4", + "UnZipFolder" + ) + } + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/ruleprocessor/RuleProcessorFactoryTest.kt b/src/test/kotlin/net/bytedance/security/app/ruleprocessor/RuleProcessorFactoryTest.kt new file mode 100644 index 0000000..2fdcace --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/ruleprocessor/RuleProcessorFactoryTest.kt @@ -0,0 +1,75 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.ruleprocessor + +import kotlinx.coroutines.* +import net.bytedance.security.app.PreAnalyzeContext +import net.bytedance.security.app.rules.IRulesForContext +import net.bytedance.security.app.rules.RulesTest +import net.bytedance.security.app.taintflow.TaintAnalyzer +import org.junit.jupiter.api.Test +import test.SootHelper +import test.TestHelper + +internal open class RuleProcessorFactoryTest { + + init { + SootHelper.initSoot( + "RuleProcessorFactoryTest", + listOf("${TestHelper.getTestClassSourceFileDirectory(this.javaClass.name)}/testdata") + ) + } + + + @Test + fun testCreateAllRules() { + runBlocking { createAllRules() } + } + + suspend fun createAllRules() { + val rules = RulesTest.createDefaultRules() + val ctx = createContext(rules) + val jobs = ArrayList() + val scope = CoroutineScope(Dispatchers.Default) + val analyzers = ArrayList() + for (r in rules.allRules) { + val rp = RuleProcessorFactory.create(ctx, r.mode) + val job = scope.launch { + println("process ${rp.javaClass.name} ${r.name}") + rp.process(r) + if (rp is TaintFlowRuleProcessor) { + synchronized(analyzers) { + analyzers.addAll(rp.analyzers) + } + } + } + jobs.add(job) + } + + jobs.joinAll() + println("analyzers: ${analyzers.size}") + } + + companion object { + suspend fun createContext(rules: IRulesForContext): PreAnalyzeContext { + val ctx = PreAnalyzeContext() + ctx.createContext(rules, true) + return ctx + } + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/ruleprocessor/SliceModeProcessorTest.kt b/src/test/kotlin/net/bytedance/security/app/ruleprocessor/SliceModeProcessorTest.kt new file mode 100644 index 0000000..8a85540 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/ruleprocessor/SliceModeProcessorTest.kt @@ -0,0 +1,102 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.ruleprocessor + +import kotlinx.coroutines.runBlocking +import net.bytedance.security.app.MethodFinder +import net.bytedance.security.app.PLUtils +import net.bytedance.security.app.rules.RuleFactory +import net.bytedance.security.app.rules.Rules +import net.bytedance.security.app.taintflow.TaintAnalyzer +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.BeforeEach +import org.junit.jupiter.api.Test +import test.SootHelper +import test.TestHelper + +internal class SliceModeProcessorTest { + init { + SootHelper.initSoot( + "SliceModeProcessorTest", + listOf("${TestHelper.getTestClassSourceFileDirectory(this.javaClass.name)}/testdata") + ) + } + + @BeforeEach + fun clearCache() { + MethodFinder.clearCache() + } + + @Test + fun createAnalyzersForSourceAndSink() { + val rules = Rules( + listOf( + "${TestHelper.getTestClassSourceFileDirectory(this.javaClass.name)}/testdata/unZipSlipSliceMode.json" + ), RuleFactory() + ) + + runBlocking { + rules.loadRules() + val ctx = RuleProcessorFactoryTest.createContext(rules) + PLUtils.dumpClass("net.bytedance.security.app.ruleprocessor.testdata.ZipSlip") + val rp = RuleProcessorFactory.create(ctx, rules.allRules[0].mode) + rp.process(rules.allRules[0]) + val analyzers = (rp as SliceModeProcessor).analyzers + assertEquals(3, analyzers.size) + analyzers.forEach { + assertEquals(1, it.data.sourcePointerSet.size) + assertEquals(1, it.data.sinkPointerSet.size) + } + + mustContains( + analyzers, + "->\$r3", + "UnZipFolderFix1" + ) + mustContains( + analyzers, + "->\$r3", + "UnZipFolder" + ) + mustContains( + analyzers, + "->\$r4", + "UnZipFolderFix2" + ) + } + } + + companion object { + fun mustContains(analyzers: List, sourcePtrStr: String, entry: String) { + for (analyzer in analyzers) { + if (analyzer.entryMethod.name == entry) { + try { + val srcPtr = analyzer.data.pointerIndexMap[sourcePtrStr]!! + if (analyzer.data.sourcePointerSet.contains(srcPtr)) { + return + } + } catch (ex: Exception) { + ex.printStackTrace() + } + } + + } + throw Exception("not found $entry") + } + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/ruleprocessor/testdata/ZipSlip.java b/src/test/kotlin/net/bytedance/security/app/ruleprocessor/testdata/ZipSlip.java new file mode 100644 index 0000000..5395fa9 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/ruleprocessor/testdata/ZipSlip.java @@ -0,0 +1,100 @@ +package net.bytedance.security.app.ruleprocessor.testdata; + + +import java.io.File; +import java.io.FileInputStream; +import java.io.FileOutputStream; +import java.util.zip.ZipEntry; +import java.util.zip.ZipInputStream; + +public class ZipSlip { + + public void UnZipFolder(String zipFileString, String outPathString) throws Exception { + ZipInputStream inZip = new ZipInputStream(new FileInputStream(zipFileString)); + ZipEntry zipEntry; + String szName = ""; + while ((zipEntry = inZip.getNextEntry()) != null) { + szName = zipEntry.getName(); + if (zipEntry.isDirectory()) { + szName = szName.substring(0, szName.length() - 1); + File folder = new File(outPathString + File.separator + szName); + folder.mkdirs(); + } else { + File file = new File(outPathString + File.separator + szName); + file.createNewFile(); + FileOutputStream out = new FileOutputStream(file); + int len; + byte[] buffer = new byte[1024]; + while ((len = inZip.read(buffer)) != -1) { + out.write(buffer, 0, len); + out.flush(); + } + out.close(); + } + } + inZip.close(); + } + + public void UnZipFolderFix1(String zipFileString, String outPathString) throws Exception { + ZipInputStream inZip = new ZipInputStream(new FileInputStream(zipFileString)); + ZipEntry zipEntry; + String szName = ""; + while ((zipEntry = inZip.getNextEntry()) != null) { + szName = zipEntry.getName(); +// if(szName.contains("..")){ +// throw new SecurityException("unzip slip"); +// } + + if (zipEntry.isDirectory()) { + szName = szName.substring(0, szName.length() - 1); + File folder = new File(outPathString + File.separator + szName); + folder.mkdirs(); + } else { + File file = new File(outPathString + File.separator + szName); + file.createNewFile(); + FileOutputStream out = new FileOutputStream(file); + int len; + byte[] buffer = new byte[1024]; + while ((len = inZip.read(buffer)) != -1) { + out.write(buffer, 0, len); + out.flush(); + } + out.close(); + } + } + inZip.close(); + } + + + public void UnZipFolderFix2(String zipFileString, String outPathString) throws Exception { + ZipInputStream inZip = new ZipInputStream(new FileInputStream(zipFileString)); + ZipEntry zipEntry; + String szName = ""; + while ((zipEntry = inZip.getNextEntry()) != null) { + szName = zipEntry.getName(); + File tmp = new File(outPathString + File.separator + szName); + if (!tmp.getCanonicalPath().startsWith(outPathString)) { + throw new SecurityException("unzip slip"); + } + + if (zipEntry.isDirectory()) { + szName = szName.substring(0, szName.length() - 1); + File folder = new File(outPathString + File.separator + szName); + folder.mkdirs(); + } else { + File file = new File(outPathString + File.separator + szName); + file.createNewFile(); + FileOutputStream out = new FileOutputStream(file); + int len; + byte[] buffer = new byte[1024]; + while ((len = inZip.read(buffer)) != -1) { + out.write(buffer, 0, len); + out.flush(); + } + out.close(); + } + } + inZip.close(); + } +} + diff --git a/src/test/kotlin/net/bytedance/security/app/ruleprocessor/testdata/unZipSlipDirectMode.json b/src/test/kotlin/net/bytedance/security/app/ruleprocessor/testdata/unZipSlipDirectMode.json new file mode 100644 index 0000000..1a2af21 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/ruleprocessor/testdata/unZipSlipDirectMode.json @@ -0,0 +1,31 @@ +{ + "unZipSlipDirectModeMode": { + "DirectMode": true, + "traceDepth": 8, + "desc": { + "name": "unZipSlip" + }, + "entry": { + "methods": [ + "" + ] + }, + "source": { + "Return": [ + "" + ] + }, + "sink": { + "(*)>": { + "TaintCheck": [ + "p*" + ] + }, + "(*)>": { + "TaintCheck": [ + "p*" + ] + } + } + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/ruleprocessor/testdata/unZipSlipSliceMode.json b/src/test/kotlin/net/bytedance/security/app/ruleprocessor/testdata/unZipSlipSliceMode.json new file mode 100644 index 0000000..f2ebc2f --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/ruleprocessor/testdata/unZipSlipSliceMode.json @@ -0,0 +1,28 @@ +{ + "unZipSlipSliceMode": { + "SliceMode": true, + "traceDepth": 8, + "desc": { + "name": "unZipSlip" + }, + "entry": { + }, + "source": { + "Return": [ + "" + ] + }, + "sink": { + "(*)>": { + "TaintCheck": [ + "p*" + ] + }, + "(*)>": { + "TaintCheck": [ + "p*" + ] + } + } + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/rules/RulesTest.kt b/src/test/kotlin/net/bytedance/security/app/rules/RulesTest.kt new file mode 100644 index 0000000..b5cfa26 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/rules/RulesTest.kt @@ -0,0 +1,99 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.rules + +import kotlinx.coroutines.runBlocking +import net.bytedance.security.app.getConfig +import org.junit.jupiter.api.Test +import java.io.File + +internal class RulesTest { + + + fun createDefaultRules(): Rules { + val rules = Rules( + listOf( + "${getConfig().rulePath}/unZipSlip.json", + ), + RuleFactory() + ) + runBlocking { + rules.loadRules() + } + return rules + } + + @Test + fun constStringPatterns() { + val rules = createDefaultRules() + println(rules.constStringPatterns().toSortedSet().toList()) + } + + @Test + fun newInstances() { + val rules = createDefaultRules() + println(rules.newInstances().toSortedSet().toList()) + } + + @Test + fun fields() { + val rules = createDefaultRules() + println(rules.fields().toSortedSet().toList()) + } + + + @Test + fun testAllRules() { + val rules = Rules( + getAllRules(), + RuleFactory() + ) + runBlocking { + rules.loadRules() + } + println("const strings=${rules.constStringPatterns().toSortedSet().toList()}") + println("fields=${rules.fields().toSortedSet().toList()}") + println("new instances=${rules.newInstances().toSortedSet().toList()}") + + } + + companion object { + fun getAllRules(): List { + val rules = ArrayList() + File(getConfig().rulePath).walk().forEach { +// println(it.absolutePath) + if (it.absolutePath.endsWith(".json") || it.absolutePath.endsWith(".json5")) { + rules.add(it.absolutePath) + } + } + println(rules) + return rules + } + + fun createDefaultRules(): Rules { + val rules = Rules( + getAllRules(), + RuleFactory() + ) + runBlocking { + rules.loadRules() + } + return rules + } + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/sanitizer/SanitizerFactoryTest.kt b/src/test/kotlin/net/bytedance/security/app/sanitizer/SanitizerFactoryTest.kt new file mode 100644 index 0000000..5495eb4 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/sanitizer/SanitizerFactoryTest.kt @@ -0,0 +1,230 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.sanitizer + +import kotlinx.coroutines.runBlocking +import kotlinx.serialization.json.Json +import kotlinx.serialization.json.jsonPrimitive +import net.bytedance.security.app.MethodFinder +import net.bytedance.security.app.pointer.PLLocalPointer +import net.bytedance.security.app.ruleprocessor.RuleProcessorFactoryTest.Companion.createContext +import net.bytedance.security.app.rules.DirectModeRule +import net.bytedance.security.app.rules.RuleFactory +import net.bytedance.security.app.rules.Rules +import net.bytedance.security.app.taintflow.TwoStagePointerAnalyzeTest +import org.junit.jupiter.api.Assertions.* +import org.junit.jupiter.api.BeforeEach +import org.junit.jupiter.api.Test +import soot.RefType +import soot.Scene +import soot.jimple.LongConstant +import soot.jimple.NullConstant +import soot.jimple.StringConstant +import test.SootHelper +import test.TestHelper + +internal class SanitizerFactoryTest { + init { + SootHelper.initSoot( + "SanitizerFactoryTest", + listOf("${TestHelper.getTestClassSourceFileDirectory(this.javaClass.name)}/testdata") + ) + } + + @BeforeEach + fun clearCache() { + SanitizerFactory.clearCache() + MethodFinder.clearCache() + } + + @Test + fun createConstStringSanitizers() { + val rules = Rules( + listOf( + "${TestHelper.getTestClassSourceFileDirectory(this.javaClass.name)}/testdata/unZipSlipConstStringSanitizer.json" + ), RuleFactory() + ) + runBlocking { + rules.loadRules() + } + val taintedRule = rules.allRules[0] as DirectModeRule + val strs = taintedRule.constStringPatterns() + assert(strs.size == 1) + runBlocking { + val ctx = createContext(rules) + val sanitizers = SanitizerFactory.createSanitizers(taintedRule, ctx) + assertTrue(sanitizers.size == 1) + val s0 = sanitizers[0] + assertTrue(s0 is ConstStringCheckSanitizer) + println("sanitizers=${(s0 as ConstStringCheckSanitizer).constStrings}") + assertTrue(sanitizersResult(sanitizers, getUnzipFolderSrc())) + } + } + + fun getUnzipFolderSrc(): PLLocalPointer { + val m = Scene.v() + .getMethod("") + return PLLocalPointer(m, "\$r3", RefType.v("java.lang.String")) + } + + fun getUnZipFolderFix1Src(): PLLocalPointer { + val m = Scene.v() + .getMethod("") + return PLLocalPointer(m, "\$r3", RefType.v("java.lang.String")) + } + + fun sanitizersResult(sanitizers: List, src: PLLocalPointer): Boolean { + val entry = Scene.v().getMethod("") + val tsp = TwoStagePointerAnalyzeTest.createDefaultTwoStagePointerAnalyze(entry) + runBlocking { + tsp.doPointerAnalyze() + } + for (s in sanitizers) { + if (s.matched(SanitizeContext(tsp.ctx, src))) { + return true + } + } + return false + } + + @Test + fun createFieldSanitizers() { + val rules = Rules( + listOf( + "${TestHelper.getTestClassSourceFileDirectory(this.javaClass.name)}/testdata/unZipSlipFieldSanitizer.json" + ), RuleFactory() + ) + runBlocking { + rules.loadRules() + } + val taintedRule = rules.allRules[0] as DirectModeRule + val strs = taintedRule.fields() + assert(strs.size == 1) + runBlocking { + val ctx = createContext(rules) +// PLUtils.DumpClass("net.bytedance.security.app.sanitizer.testdata.ZipSlip") + val sanitizers = SanitizerFactory.createSanitizers(taintedRule, ctx) + assertTrue(sanitizers.size == 1) + val s0 = sanitizers[0] + assertTrue(s0 is TaintCheckSanitizer) +// assertTrue((s0 as ConstStringCheckSanitizer).consts.size == 1) + val taints = (s0 as TaintCheckSanitizer).taints + assertTrue(taints.size == 1) + assertTrue(taints.first().method.name == "UnZipFolder") + println("sanitizers=${taints}") + assertFalse(sanitizersResult(sanitizers, getUnZipFolderFix1Src())) + } + } + + @Test + fun createMethod1() { + val rules = Rules( + listOf( + "${TestHelper.getTestClassSourceFileDirectory(this.javaClass.name)}/testdata/unZipSlipMethodCheck1.json" + ), RuleFactory() + ) + runBlocking { + rules.loadRules() + } + val taintedRule = rules.allRules[0] as DirectModeRule + runBlocking { + val ctx = createContext(rules) +// PLUtils.DumpClass("net.bytedance.security.app.sanitizer.testdata.ZipSlip") + val sanitizers = SanitizerFactory.createSanitizers(taintedRule, ctx) + assertEquals(sanitizers.size, 1) + val s0 = sanitizers[0] + assertTrue(s0 is SanitizeOrRules) + val so = s0 as SanitizeOrRules +// assertTrue((s0 as ConstStringCheckSanitizer).consts.size == 1) + assertEquals(so.rules.size, 1) + val tcs = so.rules[0] as TaintCheckSanitizer + assertEquals(tcs.taints.size, 1) + assertEquals(tcs.taints.first().method.name, "UnZipFolderFix1") + assertTrue(tcs.notTaints.isEmpty()) + assertTrue(tcs.constStrings.isEmpty()) + + assertFalse(sanitizersResult(sanitizers, getUnzipFolderSrc())) + assertTrue(sanitizersResult(sanitizers, getUnZipFolderFix1Src())) + } + } + + + @Test + fun createMethod4() { + val rules = Rules( + listOf( + "${TestHelper.getTestClassSourceFileDirectory(this.javaClass.name)}/testdata/unZipSlipMethodCheck4.json" + ), RuleFactory() + ) + runBlocking { + rules.loadRules() + } + val taintedRule = rules.allRules[0] as DirectModeRule + runBlocking { + val ctx = createContext(rules) +// PLUtils.DumpClass("net.bytedance.security.app.sanitizer.testdata.ZipSlip") + val sanitizers = SanitizerFactory.createSanitizers(taintedRule, ctx) + assertEquals(sanitizers.size, 1) + val s0 = sanitizers[0] + assertTrue(s0 is MethodCheckSanitizer) + val mc = s0 as MethodCheckSanitizer + assertEquals(mc.methods.size, 1) + assertTrue(sanitizersResult(sanitizers, getUnzipFolderSrc())) + assertTrue(sanitizersResult(sanitizers, getUnZipFolderFix1Src())) + } + } + + @Test + fun testConstant() { + val str = StringConstant.v("aaa") + assertEquals("aaa", str.getStringValue()) + val n = LongConstant.v(3) + assertEquals("3", n.getStringValue()) + val nullStr = NullConstant.v() + assertEquals(nullStr.getStringValue(), "null") + } + + @Test + fun jsonConstant() { + val n = Json.parseToJsonElement("1") + assertEquals(n.jsonPrimitive.content, "1") + val str = Json.parseToJsonElement("\"aaa\"") + assertEquals(str.jsonPrimitive.content, "aaa") + val nullStr = Json.parseToJsonElement("null") + assertEquals(nullStr.jsonPrimitive.content, "null") + } + + data class Table(val pattern: String, val target: String, val result: Boolean) + + @Test + fun isSanitizeStrMatch() { + val samples = listOf( + Table("12", "12", true), + Table("12*", "12", true), + Table("*12", "12", true), + Table("67108864:&", "67108865", true), + Table("67108864:&", "123456", false), + Table("67108864:|", "67108865", false), + Table("67108864:|", "123456", true), + ) + for (s in samples) { + val r = TaintCheckSanitizer.isSanitizeStrMatch(s.pattern, s.target) + assertEquals(s.result, r, s.pattern + "," + s.target) + } + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/sanitizer/testdata/ZipSlip.java b/src/test/kotlin/net/bytedance/security/app/sanitizer/testdata/ZipSlip.java new file mode 100644 index 0000000..24890b2 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/sanitizer/testdata/ZipSlip.java @@ -0,0 +1,87 @@ +package net.bytedance.security.app.sanitizer.testdata; + + +import java.io.File; +import java.io.FileInputStream; +import java.io.FileOutputStream; +import java.util.zip.ZipEntry; +import java.util.zip.ZipInputStream; + +public class ZipSlip { + private String instanceField = "aaa"; + private static String staticField = "staticField"; + + public void UnZipFolder(String zipFileString, String outPathString) throws Exception { + ZipInputStream inZip = new ZipInputStream(new FileInputStream(zipFileString)); + ZipEntry zipEntry; + String szName = ""; + while ((zipEntry = inZip.getNextEntry()) != null) { + szName = zipEntry.getName(); + if (zipEntry.isDirectory()) { + szName = szName.substring(0, szName.length() - 1); + File folder = new File(outPathString + File.separator + staticField + szName + "constString"); + folder.mkdirs(); + } else { + File file = new File(outPathString + File.separator + instanceField + szName); + file.createNewFile(); + FileOutputStream out = new FileOutputStream(file); + int len; + byte[] buffer = new byte[1024]; + while ((len = inZip.read(buffer)) != -1) { + out.write(buffer, 0, len); + out.flush(); + } + out.close(); + } + } + inZip.close(); + } + + public void UnZipFolderFix1(String zipFileString, String outPathString) throws Exception { + ZipInputStream inZip = new ZipInputStream(new FileInputStream(zipFileString)); + ZipEntry zipEntry; + String szName = ""; + while ((zipEntry = inZip.getNextEntry()) != null) { + szName = zipEntry.getName(); + if (szName.contains("..")) { + throw new SecurityException("unzip slip"); + } + + if (zipEntry.isDirectory()) { + szName = szName.substring(0, szName.length() - 1); + File folder = new File(outPathString + File.separator + szName); + folder.mkdirs(); + } else { + File file = new File(outPathString + File.separator + szName); + file.createNewFile(); + FileOutputStream out = new FileOutputStream(file); + int len; + byte[] buffer = new byte[1024]; + while ((len = inZip.read(buffer)) != -1) { + out.write(buffer, 0, len); + out.flush(); + } + out.close(); + } + } + inZip.close(); + } + + public void f() throws Exception { + UnZipFolder("path1", "outpath1"); + UnZipFolderFix1("path2", "outpath2"); + } + + public String returnStringConst() { + return "aaa"; + } + + public String returnString() { + return staticField; + } + + public void printString() { + System.out.println(returnString()); + } +} + diff --git a/src/test/kotlin/net/bytedance/security/app/sanitizer/testdata/unZipSlipConstStringSanitizer.json b/src/test/kotlin/net/bytedance/security/app/sanitizer/testdata/unZipSlipConstStringSanitizer.json new file mode 100644 index 0000000..01c6fc4 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/sanitizer/testdata/unZipSlipConstStringSanitizer.json @@ -0,0 +1,33 @@ +{ + "unZipSlipConstStringSanitizer": { + "SliceMode": true, + "traceDepth": 8, + "desc": { + "name": "unZipSlipConstStringSanitizer" + }, + "entry": { + "methods": [ + "" + ] + }, + "source": { + "Return": [ + "" + ] + }, + "sink": { + "(*)>": { + "TaintCheck": [ + "p*" + ] + } + }, + "sanitize": { + "rule2": { + "ConstString": [ + "..*" + ] + } + } + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/sanitizer/testdata/unZipSlipFieldSanitizer.json b/src/test/kotlin/net/bytedance/security/app/sanitizer/testdata/unZipSlipFieldSanitizer.json new file mode 100644 index 0000000..d5d4307 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/sanitizer/testdata/unZipSlipFieldSanitizer.json @@ -0,0 +1,35 @@ +{ + "unZipSlipFieldSanitizer": { + "SliceMode": true, + "traceDepth": 8, + "desc": { + "name": "unZipSlipFieldSanitizer" + }, + "entry": { + "methods": [ + "" + ] + }, + "source": { + "Return": [ + "" + ] + }, + "sink": { + "(*)>": { + "TaintCheck": [ + "p*" + ] + } + }, + "sanitize": { + "rule2": { + "": { + "TaintCheck": [ + "@this" + ] + } + } + } + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/sanitizer/testdata/unZipSlipMethodCheck1.json b/src/test/kotlin/net/bytedance/security/app/sanitizer/testdata/unZipSlipMethodCheck1.json new file mode 100644 index 0000000..2f32dac --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/sanitizer/testdata/unZipSlipMethodCheck1.json @@ -0,0 +1,38 @@ +{ + "unZipSlipMethodCheck1": { + "SliceMode": true, + "traceDepth": 8, + "desc": { + "name": "unZipSlipMethodCheck1" + }, + "entry": { + "methods": [ + "" + ] + }, + "source": { + "Return": [ + "" + ] + }, + "sink": { + "(*)>": { + "TaintCheck": [ + "p*" + ] + } + }, + "sanitize": { + "containsDotDot": { + "": { + "TaintCheck": [ + "@this" + ], + "p0": [ + "..*" + ] + } + } + } + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/sanitizer/testdata/unZipSlipMethodCheck4.json b/src/test/kotlin/net/bytedance/security/app/sanitizer/testdata/unZipSlipMethodCheck4.json new file mode 100644 index 0000000..fa888cf --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/sanitizer/testdata/unZipSlipMethodCheck4.json @@ -0,0 +1,32 @@ +{ + "unZipSlipMethodCheck4": { + "SliceMode": true, + "traceDepth": 8, + "desc": { + "name": "unZipSlipMethodCheck4" + }, + "entry": { + "methods": [ + "" + ] + }, + "source": { + "Return": [ + "" + ] + }, + "sink": { + "(*)>": { + "TaintCheck": [ + "p*" + ] + } + }, + "sanitize": { + "containsDotDot": { + "": { + } + } + } + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/sanitizer/testdata/unZipSlipSliceMode2.json b/src/test/kotlin/net/bytedance/security/app/sanitizer/testdata/unZipSlipSliceMode2.json new file mode 100644 index 0000000..c81d62a --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/sanitizer/testdata/unZipSlipSliceMode2.json @@ -0,0 +1,64 @@ +{ + "unZipSlipSliceMode2": { + "SliceMode": true, + "traceDepth": 8, + "desc": { + "name": "unZipSlipSliceMode2" + }, + "entry": { + }, + "source": { + "Return": [ + "" + ] + }, + "sink": { + "(*)>": { + "TaintCheck": [ + "p*" + ] + } + }, + "sanitize": { + "rule1": { + "": { + "TaintCheck":["@this"] + } + }, + + "containsDotDot": { + "": { + "TaintCheck": ["@this"], + "p0":["..*"] + } + }, + "kotlinContainsDotdot": { + "<*: boolean contains$default(java.lang.CharSequence,java.lang.CharSequence,boolean,int,java.lang.Object)>":{ + "TaintCheck": ["p0"], + "p1": ["..*"] + } + }, + "indexDotDot": { + "": { + "TaintCheck": ["@this"], + "p0":["..*"] + } + }, + "kotlinIndexDotdot": { + "<*: int indexOf$default(java.lang.CharSequence,java.lang.String,int,boolean,int,java.lang.Object)>": { + "TaintCheck": [ + "p0" + ], + "p1": [ + "..*" + ] + } + }, + "lastIndexOf": { + "": {}, + "": { + "TaintCheck": ["@this"] + } + } + } +}, \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/taintflow/AnalyzeContextTest.kt b/src/test/kotlin/net/bytedance/security/app/taintflow/AnalyzeContextTest.kt new file mode 100644 index 0000000..7b0bd71 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/taintflow/AnalyzeContextTest.kt @@ -0,0 +1,52 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.taintflow + +import net.bytedance.security.app.PLUtils +import net.bytedance.security.app.pointer.PLLocalPointer +import net.bytedance.security.app.pointer.PLPointer +import net.bytedance.security.app.pointer.PointerFactory +import org.junit.jupiter.api.Test +import soot.UnknownType +import test.SootHelper +import test.TestHelper + +internal class AnalyzeContextTest { + val ctx = AnalyzeContext(PointerFactory()) + + init { + SootHelper.initSoot( + "AnalyzeContextTest", + listOf("${TestHelper.getTestClassSourceFileDirectory(this.javaClass.name)}/testdata") + ) + PLUtils.createCustomClass() + } + + @Test + fun collectPropagation() { + val p1 = newNode(1) + ctx.collectPropagation(p1, true) + ctx.collectPropagation(p1, true) + } + + companion object { + fun newNode(name: Int): PLPointer { + return PLLocalPointer(TwoStagePointerAnalyze.getPseudoEntryMethod(), name.toString(), UnknownType.v()) + } + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/taintflow/TaintTweakTaintFlowRuleTest.kt b/src/test/kotlin/net/bytedance/security/app/taintflow/TaintTweakTaintFlowRuleTest.kt new file mode 100644 index 0000000..7174754 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/taintflow/TaintTweakTaintFlowRuleTest.kt @@ -0,0 +1,56 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.taintflow + +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Test + +internal class TaintTweakTaintFlowRuleTest { + + @Test + fun flow() { + val m1 = mapOf( + 1 to listOf(1, 2, 3), + 2 to listOf(2, 3, 4), + 4 to listOf(8) + ) + val m2 = mapOf( + 1 to listOf(1), + 2 to listOf(7), + 3 to listOf(9) + ) + val m3 = m2 + m1 + assertEquals( + m3, mapOf( + 1 to listOf(1, 2, 3), + 2 to listOf(2, 3, 4), + 4 to listOf(8), + 3 to listOf(9) + ) + ) + val m4 = m1 + m2 + assertEquals( + m4, mapOf( + 1 to listOf(1), + 2 to listOf(7), + 3 to listOf(9), + 4 to listOf(8) + ) + ) + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/taintflow/TwoStagePointerAnalyzeTest.kt b/src/test/kotlin/net/bytedance/security/app/taintflow/TwoStagePointerAnalyzeTest.kt new file mode 100644 index 0000000..a51f513 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/taintflow/TwoStagePointerAnalyzeTest.kt @@ -0,0 +1,180 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.taintflow + +import kotlinx.coroutines.runBlocking +import net.bytedance.security.app.PLUtils +import net.bytedance.security.app.PreAnalyzeContext +import net.bytedance.security.app.engineconfig.EngineConfig +import net.bytedance.security.app.pathfinder.TaintPathFinder +import net.bytedance.security.app.pointer.PointerFactory +import net.bytedance.security.app.preprocess.AnalyzePreProcessor +import net.bytedance.security.app.preprocess.MethodFieldConstCacheVisitor +import net.bytedance.security.app.preprocess.MethodSSAVisitor +import net.bytedance.security.app.preprocess.MethodStmtFieldCache +import org.junit.jupiter.api.Assertions.assertTrue +import org.junit.jupiter.api.Test +import soot.Scene +import soot.SootMethod +import soot.UnknownType +import test.SootHelper +import test.TestHelper + +internal class TwoStagePointerAnalyzeTest { + init { + val ctx = PreAnalyzeContext() + val cam = AnalyzePreProcessor(10, ctx) + + SootHelper.initSoot( + "TwiStagePointerAnalyzeTest", + listOf("${TestHelper.getTestClassSourceFileDirectory(this.javaClass.name)}/testdata") + ) + cam.addMethodVisitor { + MethodSSAVisitor() + }.addMethodVisitor { + MethodFieldConstCacheVisitor( + ctx, + MethodStmtFieldCache(), + HashSet(), HashSet(), HashSet() + ) + } + runBlocking { cam.run() } + } + + + @Test + fun solver() { + val entry = Scene.v() + .getMethod("") + val tsp = createDefaultTwoStagePointerAnalyze(entry) + runBlocking { + tsp.doPointerAnalyze() + } + PLUtils.dumpClass("net.bytedance.security.app.taintflow.testdata.ZipSlip") + val srcName = "\$r3" + val sinkName = "\$r21" + val srcPtr = tsp.pt.allocLocal(entry, srcName, UnknownType.v()) + val sinkPtr = tsp.pt.allocLocal(entry, sinkName, UnknownType.v()) + val allTaint = tsp.ctx.collectPropagation(srcPtr, false) + assertTrue(allTaint.contains(sinkPtr)) + println(tsp.ctx.dump()) + val path = + TaintPathFinder.bfsSearch(srcPtr, setOf(sinkPtr), tsp.ctx.variableFlowGraph, 256, "test") + ?.map { + it.signature() + } + println("path=$path") + } + + @Test + fun solverStaticMethod() { + PLUtils.dumpClass("net.bytedance.security.app.taintflow.testdata.TaintExample") + val entry = Scene.v() + .getMethod("") + val pt = PointerFactory() + val tsp = createDefaultTwoStagePointerAnalyze(entry) + runBlocking { + tsp.doPointerAnalyze() + } + + val srcPtr = pt.allocLocal( + Scene.v() + .getMethod(""), + "\$r0", + UnknownType.v() + ) + val sinkPtr = pt.allocLocal(entry, "\$r0", UnknownType.v()) + val allTaint = tsp.ctx.collectPropagation(srcPtr, false) + assertTrue(allTaint.contains(sinkPtr)) + println(tsp.ctx.dump()) + } + + @Test + fun solverInstanceMethod() { + PLUtils.dumpClass("net.bytedance.security.app.taintflow.testdata.TaintExample") + val entry = Scene.v() + .getMethod("") + val pt = PointerFactory() + val tsp = createDefaultTwoStagePointerAnalyze(entry) + runBlocking { + tsp.doPointerAnalyze() + } + + val srcPtr = pt.allocLocal( + Scene.v() + .getMethod(""), + "\$r0", + UnknownType.v() + ) + val falseSinkPtr = pt.allocLocal(entry, "\$r0", UnknownType.v()) + val allTaint = tsp.ctx.collectPropagation(srcPtr, false) + assertTrue(!allTaint.contains(falseSinkPtr)) + val sinkPtr = pt.allocLocal(entry, "\$r1", UnknownType.v()) + assertTrue(tsp.ctx.collectPropagation(srcPtr, false).contains(sinkPtr)) + } + + @Test + fun flowCrossMethodFalsePositiveExample() { + PLUtils.dumpClass("net.bytedance.security.app.taintflow.testdata.TaintExample") + val entry = Scene.v() + .getMethod("") + val sinkMethod = + Scene.v().getMethod("") + val notSinkMethod = + Scene.v().getMethod("") + + val pt = PointerFactory() + val tsp = createDefaultTwoStagePointerAnalyze(entry) + runBlocking { + tsp.doPointerAnalyze() + } + + val srcPtr = pt.allocLocal(entry, "\$r0", UnknownType.v()) + val falseSrcPtr = pt.allocLocal(entry, "\$r3", UnknownType.v()) + val sinkPtr = pt.allocLocal(sinkMethod, "@parameter0", UnknownType.v()) + val falseSinkPtr = pt.allocLocal(notSinkMethod, "@parameter0", UnknownType.v()) + + + val srcTainted = tsp.ctx.collectPropagation(srcPtr, false) + val falseSrcTainted = tsp.ctx.collectPropagation(falseSrcPtr, false) + + assertTrue(srcTainted.contains(falseSinkPtr)) + assertTrue(srcTainted.contains(sinkPtr)) + assertTrue(falseSrcTainted.contains(falseSinkPtr)) + assertTrue(falseSrcTainted.contains(sinkPtr)) + } + + companion object { + + fun createDefaultTwoStagePointerAnalyze(entry: SootMethod): TwoStagePointerAnalyze { + val tsp = TwoStagePointerAnalyze( + entry.signature, + entry, + AnalyzeContext(PointerFactory()), + 3, + DefaultPointerPropagationRule(EngineConfig.PointerPropagationConfig), + DefaultVariableFlowRule(EngineConfig.variableFlowConfig), + DefaultMethodAnalyzeMode, + 100000 + ) + return tsp + } + + } + +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/taintflow/testdata/Taint.java b/src/test/kotlin/net/bytedance/security/app/taintflow/testdata/Taint.java new file mode 100644 index 0000000..512fab6 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/taintflow/testdata/Taint.java @@ -0,0 +1,15 @@ +package net.bytedance.security.app.taintflow.testdata; + +public class Taint { + public static Object source() { + return new Object(); + } + + public static void sink(Object object) { + } + public static void notSink(Object object) { + } + public static Object sanitize(Object object) { + return object; + } +} diff --git a/src/test/kotlin/net/bytedance/security/app/taintflow/testdata/TaintExample.java b/src/test/kotlin/net/bytedance/security/app/taintflow/testdata/TaintExample.java new file mode 100644 index 0000000..a52b360 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/taintflow/testdata/TaintExample.java @@ -0,0 +1,38 @@ +package net.bytedance.security.app.taintflow.testdata; + +public class TaintExample { + static void TaintCrossStaticMethod() { + Taint.sink(staticSource2()); + } + + static Object staticSource1() { + return Taint.source(); + } + + static Object staticSource2() { + return staticSource1(); + } + + void TaintCrossInstanceMethod() { + Taint.sink(instanceSource2()); + } + + Object instanceSource1() { + return Taint.source(); + } + + Object instanceSource2() { + return instanceSource1(); + } + Object through(Object arg){ + return arg; + } + void flowCrossMethod(){ + Object source=Taint.source(); + Object o1=through(source); + Taint.notSink(o1); + Object s2=new Object(); + Object o2=through(s2); + Taint.sink(o2); + } +} diff --git a/src/test/kotlin/net/bytedance/security/app/taintflow/testdata/ZipSlip.java b/src/test/kotlin/net/bytedance/security/app/taintflow/testdata/ZipSlip.java new file mode 100644 index 0000000..c6bda05 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/taintflow/testdata/ZipSlip.java @@ -0,0 +1,78 @@ +package net.bytedance.security.app.taintflow.testdata; + + +import java.io.File; +import java.io.FileInputStream; +import java.io.FileOutputStream; +import java.util.zip.ZipEntry; +import java.util.zip.ZipInputStream; + +public class ZipSlip { + private String instanceField = "aaa"; + private static String staticField = "staticField"; + + public void UnZipFolder(String zipFileString, String outPathString) throws Exception { + ZipInputStream inZip = new ZipInputStream(new FileInputStream(zipFileString)); + ZipEntry zipEntry; + String szName = ""; + while ((zipEntry = inZip.getNextEntry()) != null) { + szName = zipEntry.getName(); + if (zipEntry.isDirectory()) { + szName = szName.substring(0, szName.length() - 1); + File folder = new File(outPathString + File.separator + staticField + szName + "constString"); + folder.mkdirs(); + } else { + File file = new File(outPathString + File.separator + instanceField + szName); + file.createNewFile(); + FileOutputStream out = new FileOutputStream(file); + int len; + byte[] buffer = new byte[1024]; + while ((len = inZip.read(buffer)) != -1) { + out.write(buffer, 0, len); + out.flush(); + } + out.close(); + } + } + inZip.close(); + } + + public void UnZipFolderFix1(String zipFileString, String outPathString) throws Exception { + ZipInputStream inZip = new ZipInputStream(new FileInputStream(zipFileString)); + ZipEntry zipEntry; + String szName = ""; + while ((zipEntry = inZip.getNextEntry()) != null) { + szName = zipEntry.getName(); +// if(szName.contains("..")){ +// throw new SecurityException("unzip slip"); +// } + + if (zipEntry.isDirectory()) { + szName = szName.substring(0, szName.length() - 1); + File folder = new File(outPathString + File.separator + szName); + folder.mkdirs(); + } else { + File file = new File(outPathString + File.separator + szName); + file.createNewFile(); + FileOutputStream out = new FileOutputStream(file); + int len; + byte[] buffer = new byte[1024]; + while ((len = inZip.read(buffer)) != -1) { + out.write(buffer, 0, len); + out.flush(); + } + out.close(); + } + } + inZip.close(); + } + + public String returnString() { + return staticField; + } + + public void printString() { + System.out.println(returnString()); + } +} + diff --git a/src/test/kotlin/net/bytedance/security/app/ui/HtmlWriterTest.kt b/src/test/kotlin/net/bytedance/security/app/ui/HtmlWriterTest.kt new file mode 100644 index 0000000..fc19403 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/ui/HtmlWriterTest.kt @@ -0,0 +1,68 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.ui + +import net.bytedance.security.app.RuleData +import net.bytedance.security.app.util.Json +import org.junit.jupiter.api.Test + +val data = """ + { + //it's ok to have a comment + "unZipSlipDirectModeMode": { + "traceDepth": 8, + "desc": { + "name": "unZipSlip" + }, + "entry": { + "methods": [ + "" + ] + }, + "source": { + "Return": [ + "" + ] + }, + "sink": { + "(*)>": { + "TaintCheck": [ + "p*" + ] + }, + "(*)>": { + "TaintCheck": [ + "p*" + ] + } + } + } + } +""".trimIndent() + +internal class HtmlWriterTest { + @Test + fun testHtml() { + val s = data + val rules: Map = Json.decodeFromString(s) + val desc = rules["unZipSlipDirectModeMode"]!!.desc + val hw = HtmlWriter(desc) + val s2 = hw.generateHtml() + println(s2) + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/util/DebugKtTest.kt b/src/test/kotlin/net/bytedance/security/app/util/DebugKtTest.kt new file mode 100644 index 0000000..d34ad16 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/util/DebugKtTest.kt @@ -0,0 +1,39 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.util + +import org.junit.jupiter.api.Test + +internal class DebugKtTest { + + @Test + fun toSorted() { + val s = setOf( + "", + " ", + " ", + " ", + " " + ) + val s2 = toSorted(s) + val s3 = s.toSortedSet() + println(s2) + println(s3) + + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/util/HelperKtTest.kt b/src/test/kotlin/net/bytedance/security/app/util/HelperKtTest.kt new file mode 100644 index 0000000..b86b5f5 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/util/HelperKtTest.kt @@ -0,0 +1,53 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.util + +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Test + +internal class HelperKtTest { + @Test + fun testMethodSignatureDestruction() { + val fd = methodSignatureDestruction( + "", + ) + assertEquals(fd.className, "net.bytedance.security.app.bvaa.ComponentRisk.IntentBridge") + assertEquals(fd.returnType, "void") + assertEquals(fd.functionName, "IntentBridge2") + assertEquals(fd.args, listOf("java.lang.String", "android.content.Class\$1")) + assertEquals(fd.subSignature(), "void IntentBridge2(java.lang.String,android.content.Class\$1)") + + val fd2 = methodSignatureDestruction("(*)>") + assertEquals(fd2.returnType, "*") + assertEquals(fd2.functionName, "") + assertEquals(fd2.args, listOf("*")) + } + + @Test + fun testStringArgIndex() { + val s = "p0" + assertEquals(s.argIndex(), 0) + } + + @Test + fun testGetMethodSigFromStr() { + assertEquals(getMethodSigFromStr("xxxx"), "") + assertEquals(getMethodSigFromStr("bbb"), "bbb") + assertEquals(getMethodSigFromStr("bbb>xxxx) + + @Test + fun testDecodeFromString() { + val s = "{\"obj\":[\"null\",\"1\"]}" + val s2 = Json.j5.encodeToString(Json.j5.decodeObject(s)) + println("s2=${s2}") + val list: Data = Json.format.decodeFromString(s2) + println(list) + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/util/ProfilerTest.kt b/src/test/kotlin/net/bytedance/security/app/util/ProfilerTest.kt new file mode 100644 index 0000000..85f1896 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/util/ProfilerTest.kt @@ -0,0 +1,28 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.util + +import org.junit.jupiter.api.Test + +internal class ProfilerTest { + @Test + fun testEncode() { + profiler.ApkFile = "test.apk" + println(profiler.toString()) + } +} \ No newline at end of file diff --git a/src/test/kotlin/net/bytedance/security/app/util/TaskQueueTest.kt b/src/test/kotlin/net/bytedance/security/app/util/TaskQueueTest.kt new file mode 100644 index 0000000..cdc1697 --- /dev/null +++ b/src/test/kotlin/net/bytedance/security/app/util/TaskQueueTest.kt @@ -0,0 +1,94 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package net.bytedance.security.app.util + +import kotlinx.coroutines.* +import org.junit.jupiter.api.Assertions +import org.junit.jupiter.api.Test +import java.util.concurrent.atomic.AtomicLong + +internal class TaskQueueTest { + + @Test + fun runTask() { +// val indexSum = Concurrent() + val valSum = AtomicLong() + val q = TaskQueue("test", 7) { task, _ -> + delay(100) + valSum.addAndGet(task) +// println("process $task,$index") + } + runBlocking { + val job = q.runTask() + var sum = 0 + for (i in 1 until 1000) { + sum += i + q.addTask(i.toLong()) + } + q.addTaskFinished() + job.join() + Assertions.assertEquals(sum.toLong(), valSum.get()) + } + + } + + + @Test + fun testCancel() { + runBlocking { + val startTime = System.currentTimeMillis() + val job = launch(Dispatchers.Default) { + var nextPrintTime = startTime + var i = 0 + while (i < 5) { // computation loop, just wastes CPU + // print a message twice a second + if (System.currentTimeMillis() >= nextPrintTime) { + println("job: I'm sleeping ${i++} ...") + nextPrintTime += 500L + } + yield() + } + } + delay(1300L) // delay a bit + println("main: I'm tired of waiting!") + job.cancelAndJoin() // cancels the job and waits for its completion + println("main: Now I can quit.") + } + } + + @Test + fun testRunInMilliSeconds() { + runBlocking { + val job1 = launch(Dispatchers.Default) { + delay(1000) + println("job1 finished after delay") + } + val start = System.currentTimeMillis() + var reasonTimeout = false + runInMilliSeconds(job1, 3000, "testRunInMilliSeconds") { + println("finished because of timeout") + reasonTimeout = true + } + Assertions.assertTrue(System.currentTimeMillis() - start > 1000) + Assertions.assertTrue(System.currentTimeMillis() - start < 2000) + println("runInMilliSeconds finished") + delay(4000) + Assertions.assertFalse(reasonTimeout) + } + } +} \ No newline at end of file diff --git a/src/test/kotlin/test/SootHelper.kt b/src/test/kotlin/test/SootHelper.kt new file mode 100644 index 0000000..763b3ee --- /dev/null +++ b/src/test/kotlin/test/SootHelper.kt @@ -0,0 +1,104 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package test + +import org.junit.jupiter.api.Assertions +import soot.Scene +import soot.options.Options +import java.io.File +import java.io.InputStream +import java.util.concurrent.TimeUnit + + +object SootHelper { + + @Synchronized + fun createClassFile(paths: List, target: String) { + val f = File(target) + if (f.exists()) { + return + } + runCommand("mkdir -p $target") + + var javas = "" + paths.forEach { + val f2 = File(it) + if (!f2.exists()) { + throw Exception("$it not exist") + } + javas += " $it/*.java " + } + val path = System.getProperty("user.dir") + println("Working Directory = $path") + runCommand("javac $javas -d $target") + } + + fun runCommand(cmd: String) { + println("run command: $cmd") + val result = exec(cmd, 60) + println("$result") + } + + fun exec(cmd: String?, timeOut: Int): String? { + val args = arrayOf("/bin/sh", "-c", cmd) + val p = Runtime.getRuntime().exec(args) + val res = p.waitFor(timeOut.toLong(), TimeUnit.SECONDS) + if (!res) { + return "Time out" + } + val inputStream: InputStream = p.inputStream + var result = "" + var data = inputStream.readBytes() + result += String(data) +// if (result === "") { + data = p.errorStream.readBytes() + result += String(data) +// } + return result + } + + + @Synchronized + fun initSoot(name: String, paths: List) { + //1. compile java to class + val target = "/tmp/appshark/$name" + createClassFile(paths, target) + initSootForClasses(name, target) + } + + + fun initSootForClasses(_name: String, classesFilePath: String) { + val version = System.getProperty("java.version") + Assertions.assertEquals(version, "11.0.12", "this test only works on jdk11") + soot.G.reset() + TestHelper.appsharkInit() + Options.v().set_src_prec(Options.src_prec_class) + Options.v().set_process_dir(listOf(classesFilePath)) + Options.v().set_debug(true) + Options.v().set_verbose(true) + Options.v().set_validate(false) + Options.v().set_whole_program(true) + Options.v().set_allow_phantom_refs(true) + Options.v().set_output_format(Options.output_format_jimple) + Options.v().set_keep_line_number(true) + //exclude + Options.v().set_exclude(listOf("java.*", "org.*", "sun.*", "android.*")) + Options.v().set_no_bodies_for_excluded(true) + Scene.v().loadNecessaryClasses() + } +} \ No newline at end of file diff --git a/src/test/kotlin/test/TestHelper.kt b/src/test/kotlin/test/TestHelper.kt new file mode 100644 index 0000000..e63f992 --- /dev/null +++ b/src/test/kotlin/test/TestHelper.kt @@ -0,0 +1,50 @@ +/* +* Copyright 2022 Beijing Zitiao Network Technology Co., Ltd. +* +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. +*/ + + +package test + +import net.bytedance.security.app.MethodFinder +import net.bytedance.security.app.sanitizer.SanitizerFactory +import org.junit.jupiter.api.Assertions.assertEquals +import org.junit.jupiter.api.Test + +object TestHelper { + /** + * @param className net.Bytedance.Security.App.ContextTest + * @return $wd/SRC/test/kotlin/com/security + */ + fun getTestClassSourceFileDirectory(className: String): String { + val path = System.getProperty("user.dir") + val ss = className.split(".") + var subPath = ss.slice(0..ss.size - 2).joinToString("/") + return "$path/src/test/kotlin/$subPath" + } + + fun appsharkInit() { + MethodFinder.clearCache() + SanitizerFactory.clearCache() + } + + @Test + fun testGetTestClassSourceFileDirectory() { + val path = System.getProperty("user.dir") + assertEquals( + "$path/src/test/kotlin/net/bytedance/security/app", + getTestClassSourceFileDirectory("net.bytedance.security.app.ContextTest") + ) + } +} \ No newline at end of file