Commit graph

430 commits

Author SHA1 Message Date
Stanley Good
bdcd3e3192
add save HAR addon (#6368)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2023-09-24 13:51:22 +00:00
Stanley Good
eacbb25e93
ReadHAR addon (#6283)
* moved readhar to addons and adjusted tests

* fixed tests for read_har function

* Add har export addon (#6252)

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Maximilian Hils <git@maximilianhils.com>

* fixed path issues

* added back contrib/import_har

* [autofix.ci] apply automated fixes

* fixed windows error hopefully

* [autofix.ci] apply automated fixes

* fixed ubuntu 3.10 isoformat error

* [autofix.ci] apply automated fixes

* fixed Path import

* [autofix.ci] apply automated fixes

* need to fix ubuntu 3.10 isoformat

* [autofix.ci] apply automated fixes

* replaced Z with +00:00 before isoformatting

* fixed tox lint

* moved readhar to addons and adjusted tests

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2023-07-29 18:32:23 +02:00
Stanley Good
cb1c824255
Add har export addon (#6252)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2023-07-27 19:37:06 +00:00
Stanley Good
d4ec24d85d
HAR import: request content handling (#6250)
* added handling for request content

* fixed small error in getting request content, and added test HAR file that contains a request with post data

* changed postdata.har to HAR file from reqbin

* Deterministic file_to_flows (#6257)

* made file_to_flows deterministic

* [autofix.ci] apply automated fixes

* made dummy values easier to understand

* [autofix.ci] apply automated fixes

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>

* changed postdata.har to include only one entry

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2023-07-16 14:59:48 +02:00
Stanley Good
1c8c4ab935
Deterministic file_to_flows (#6257)
* made file_to_flows deterministic

* [autofix.ci] apply automated fixes

* made dummy values easier to understand

* [autofix.ci] apply automated fixes

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2023-07-16 11:52:18 +00:00
Stanley Good
9c99777e42
http version for flow is now based on HAR file (#6248)
* http version for flow is now based on HAR file

* [autofix.ci] apply automated fixes

* http_version is now always added as correct representation, and test files are updated with correct http_version values

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2023-07-15 09:56:09 +02:00
Stanley Good
c34c86dd90
add HAR import tests (#6231)
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2023-07-11 17:17:44 +02:00
Semnodime
fd01a0aa95
Update http-reply-from-proxy.py (#6117)
* Update http-reply-from-proxy.py

Fix #6115

* Update CHANGELOG.md

Add one line to unreleased mitmproxy next

* Update CHANGELOG.md

Correct reference to PR

---------

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2023-07-02 12:55:33 +02:00
Stanley Good
47021baaa1
first draft of read har. (#6190)
* first draft of read har.

* [autofix.ci] apply automated fixes

* Fixed using max's suggestions

* [autofix.ci] apply automated fixes

* updated to snake_case

* [autofix.ci] apply automated fixes

* Made max's changes.

* [autofix.ci] apply automated fixes

* gets server address from har file now

* [autofix.ci] apply automated fixes

* added support for if serverIPAdress wasnt found

* [autofix.ci] apply automated fixes

* editted docstring and changed peername and sockname values

* fix nits

* removed unused variable, should pass tox

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2023-06-30 08:12:55 +02:00
emanuele-em
ab9429e3f6
solve #4272 docs bug (#6127) 2023-05-14 19:22:51 +02:00
Niles Ingalls
19218bfaf2
update api for sni call (#6077) 2023-04-22 12:48:24 +02:00
Maximilian Hils
c38c01fa4f
Switch to pyproject.toml (#6016)
* switch to pyproject.toml

* fix mypy

* fix pyinstaller
2023-03-26 17:40:32 +00:00
Maximilian Hils
555a2d2199
disallow bare except statements (#6017) 2023-03-26 13:57:48 +00:00
autofix-ci[bot]
51670861e6 [autofix.ci] apply automated fixes 2023-02-27 08:37:11 +01:00
Maximilian Hils
244ff35e60
fix usage of asyncio.create_task (#5928)
this fixes #5926
2023-02-13 21:45:02 +00:00
Xiao Wang
430833e3d9
Fix server addr issue in tls_passthrough example. (#5904)
* fix server peername is None issue

peername would be None, we should use other not None property as key.

* Update tls_passthrough.py

---------

Co-authored-by: Maximilian Hils <github@maximilianhils.com>
2023-02-07 09:57:24 +01:00
dependabot[bot]
dfeddcf4ad
Bump install-pinned/black from 9101a4d68e870eaaaae21c412d1d879b93c9afcb to 13c8a20eb904ba800c87f0b34ccfd932ac2ff81d (#5899)
* Bump install-pinned/black

Bumps [install-pinned/black](https://github.com/install-pinned/black) from 9101a4d68e870eaaaae21c412d1d879b93c9afcb to 13c8a20eb904ba800c87f0b34ccfd932ac2ff81d.
- [Release notes](https://github.com/install-pinned/black/releases)
- [Commits](9101a4d68e...13c8a20eb9)

---
updated-dependencies:
- dependency-name: install-pinned/black
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* [autofix.ci] apply automated fixes

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2023-02-01 23:08:09 +00:00
Sean Flanigan
5df439b7e8
Add enhancements for contrib/mitmproxywrapper.py (#5883)
* Add options for mitmweb and random port

* Clean up proxy settings on ^C

* [autofix.ci] apply automated fixes

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2023-01-18 09:02:04 +01:00
Sean Flanigan
2bf96678ae
Adapt contrib/mitmproxywrapper.py to Python 3 strings (#5866)
* Adapt mitmproxywrapper.py to Python 3 strings

This fixes errors like this one: 

`TypeError: memoryview: a bytes-like object is required, not 'str'`

* [autofix.ci] apply automated fixes

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2023-01-13 10:04:13 +01:00
gpiechnik2
12960c0494
har: don't set pages 2022-12-17 14:32:38 +01:00
autofix-ci[bot]
8c2428c9d3
[autofix.ci] apply automated fixes 2022-11-29 13:28:41 +00:00
Maximilian Hils
8361c81cdf add autofix.ci 2022-11-29 14:27:35 +01:00
Maximilian Hils
6b71066141 cleanup mitmproxy.connection, introduce dataclass-based serialization 2022-11-21 01:27:31 +01:00
Aldo Cortesi
0bbb0215c1
more mypy (#5724)
Co-authored-by: requires.io <support@requires.io>
Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2022-11-13 18:04:34 +00:00
inceabdullah
be2b9a962e
add websocket messages in HAR file (#5663)
* add websocket messages in HAR file

* use websocket_end instead of websocket_message

* remove requestID_to_entry

* remove request

* add base64-encoded data if it is binary

* har dumper: nits

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2022-10-21 02:44:01 +02:00
m4ci3k2
c0f25e0470
Make HAR files compatible with Firefox DevTools (#5666)
Firefox Developer Tools reject to load HAR files that do not have
`har.log.pages` element (see
<https://bugzilla.mozilla.org/show_bug.cgi?id=1691240>). Adding a
placeholder entry fixes it.

Co-authored-by: Maciej Zielenkiewicz <maciej.zielenkiewicz@connectpoint.pl>
2022-10-21 01:54:07 +02:00
BigSully
b2303558b2
get real url in transparent mode (#5613)
flow.request.url shows the ip address rather than host name when mitmproxy works on transparent mode
flow.request.pretty_url can show the correct url
2022-10-14 07:44:01 +02:00
daycoder
bbfb639835
Update options-simple.py (#5600)
Usage in the docstring is incorrect. Changes "--set addheader true" to "--set addheader=true"
2022-09-21 16:39:21 +02:00
Maximilian Hils
c69239bb90 switch to stdlib logging
mitmproxy previously used a homegrown logging mechanism based around
`mitmproxy.ctx.log` and the `add_log` hook. This worked well for everything
we control, but does not work outside the mitmproxy universe.
For now we have simply ignored logging in e.g. tornado or h2, but with the
upcoming introduction of mitmproxy_wireguard we now have a dependency
on some Rust/PyO3 code for which we definitely want logs, but which also
cannot easily be changed to use our homegrown logging (PyO3 does the heavy
lifting to add interoperability with stdlib logging). Long story short,
we want to introduce a log handler for stdlib logging.

Now there are two ways how such a handler could operate:

 1. We could build a handler that forwards all stdlib log events
    into our homegrown mechanism.
 2. We embrace stdlib's logging as the correct way to do things,
    and get rid of our homegrown stuff.

This PR follows the second approach by removing the `add_log` hook and
rewriting the `TermLog` and `EventStore` addons to listen for stdlib log records.
This means that all `mitmproxy.ctx.log.info` events are now simply `logging.info` etc.

One upside of this approach is that many parts of the codebase now don't depend
on the existence of `mitmproxy.ctx` and we can use off-the-shelf things like pytest's
`caplog`. We can also now better colorize log output and/or add timestamps.
2022-09-17 17:28:35 +02:00
Francisco Boni
f9899fdd44
Improve http_manipulate_cookies.py example (#5578)
Improve the example helper functions to use the get_all functions recommended
in the Header class docs for use with cookies data. This will avoid errors when
dealing with multiple headers.

cf655a664f/mitmproxy/http.py (L148)
cf655a664f/mitmproxy/http.py (L91)
2022-09-11 15:58:29 +02:00
JannisBush
6110a2a1bb
Fix wsgi HTTPS example (#5385) 2022-05-30 14:41:10 +00:00
Pedro Augusto de Paula Barbosa
4771e42b44
Fix typo in http-trailers.py (#5384) 2022-05-30 09:53:44 +02:00
Maximilian Hils
b3587b52b2 make it black! 2022-04-28 19:11:10 +02:00
Maximilian Hils
fdde9ba3b3 use Python 3.9+ typing 2022-04-26 13:53:23 +02:00
Maximilian Hils
b31ad66225 autoflake --in-place **/*.py 2022-04-26 13:52:46 +02:00
Maximilian Hils
e83ec8390a pyupgrade --py39-plus **/*.py 2022-04-26 13:51:11 +02:00
Will Monge
a863f529ab
Example: manipulate http cookies (#5278)
* added example on how to manipulate (outbound and inbound) cookies for http flows

* added entry to the CHANGELOG
2022-04-22 12:40:23 +02:00
Maximilian Hils
648d3bc0f9
add wsreplay.py, close #5012 2022-03-30 15:14:29 +02:00
Gabriel Corona
66dd158560
Add domain fronting example (#5217)
Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2022-03-29 18:24:59 +02:00
Maximilian Hils
9d1e3107e8 pyupgrade --keep-runtime-typing --py38-plus 2022-03-29 15:29:37 +02:00
Maximilian Hils
beb49ab121 don't use #noqa for imports 2022-03-29 13:51:08 +02:00
Agustin Bacigalup
33bba38b03
Add httpdump.py example (#5190)
* add httpdump.py example

* update CHANGELOG.md

* delete print() statement

* fix flake8?
2022-03-23 13:04:38 +00:00
Eldad Marciano
cba67aa94c
an ntlm upstream addon for http \ https (#5100)
* an ntlm upstream addon for http \ https

Ntlm upstream proxy for http https all http methods

* fix filename

* lint!

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2022-03-15 20:44:09 +00:00
EndUser509
a0cf273484
Merge pull request #5099 from EndUser509/save_streamed_data2
Save streamed data
2022-03-15 21:34:49 +01:00
Robert Xiao
cee4b72459 Support async hooks. Fixes #4207. 2022-02-04 17:30:20 +01:00
Maximilian Hils
96f77453cc
fix change_upstream_proxy.py example, fix #4981 (#5007) 2021-12-20 21:10:06 +00:00
Maximilian Hils
3cb87f5a2f split tls_handshake hook into client/server and success/fail variants 2021-11-22 10:23:21 +01:00
Maximilian Hils
a72f61ef57 Merge remote-tracking branch 'origin/main' into ignore-after-clienthello 2021-11-22 09:54:08 +01:00
Marius
301f8cf79d
Create search.py (#4900)
* Create search.py

* Address linter findings

* Address linter findings
2021-11-16 12:47:34 +01:00
mame82
480052f58b
Grpc contentview (#4851)
* Partial gRPC contentview prototype, not linted, no tests, not as add-on

* Linted (flake8)

* Save dev state

* Rewrote of protobuf parser, use decoding strategy, reduced rendered data. Parser uses  generators

* minor cleanup

* fix: preferred encoding was provided as function instead of value

* flake8: line length

* Backlinked message tree objects, temporary debug out

* Partial implementation of gRPC definitions. Save state to fix a cras (data invalidate in edit mode)

* hack: deal with missing exception handling for generator based content views

* gRPC/Protoparser descriptions (with test code)

* replaced manual gzip decoding with mitmproxy.net.encoding.decode

* Refactored typing imports

* Reafctoring

* distinguish request vs response definitions, separate view config from parser config

* Code cleaning, moved customized protobuf definitions to example addon

* final cleanup

* changelog

* Stubs for tests

* Fixed render_riority of addon example

* Started adding tests

* Work on tests

* mypy

* Added pseudo encoder to tests, to cover special decodings

* Example addon test added

* finalized tests, no 100 percent coverage possible, see comments un uncovered code

* minor adjustments

* fixup tests

* Typos

Co-authored-by: Maximilian Hils <git@maximilianhils.com>
2021-10-12 13:32:56 +02:00