Messages isn’t a valid keyword (anymore?) #2

New Issue

marsmathis · 2023-11-19T04:20:18+02:00

marsmathis commented

2023-11-19 04:20:18 +02:00

Whenever I call any method (for example journey_detail()), it throws an error:

File /usr/lib/python3.11/site-packages/pyrmv-0.3.4-py3.11.egg/pyrmv/classes/Journey.py:17 in init
for message in data["Messages"]["Message"]:

KeyError: 'Messages'

I looked into it and I can’t be sure but it seems like they renamed it to "Notes" and "Note" instead of "Messages" and "Message".

Whenever I call any method (for example journey_detail()), it throws an error: File /usr/lib/python3.11/site-packages/pyrmv-0.3.4-py3.11.egg/pyrmv/classes/Journey.py:17 in __init__ for message in data["Messages"]["Message"]: KeyError: 'Messages' I looked into it and I can’t be sure but it seems like they renamed it to "Notes" and "Note" instead of "Messages" and "Message".

profitroll added reference master

2023-11-19 13:07:52 +02:00

profitroll added the

labels 2023-11-19 13:08:28 +02:00

profitroll self-assigned this 2023-11-19 13:08:32 +02:00

profitroll commented

2023-11-19 13:19:25 +02:00

"Notes" doesn't seem to be a replacement, it's an addition. It seems like "Messages" became optional at this point. RMV does not disclose response models anymore and did not update their HAFAS installation documentation, so I'll try to guess how to work around this problem...

`"Notes"` doesn't seem to be a replacement, it's an addition. It seems like `"Messages"` became optional at this point. RMV does not disclose response models anymore and did not update their HAFAS installation documentation, so I'll try to guess how to work around this problem...

profitroll referenced this issue from a commit

2023-11-19 13:19:57 +02:00

Probably a fix for #2

profitroll commented

2023-11-19 13:31:09 +02:00

Please update your pyrmv to 0.3.5 using PyPi or Gitea's package index and review if the problem is gone.

I assume that RMV did more changes to their API than those found at the moment, but without their HAFAS docs it's hard to quickly figure out which exactly.

Please update your `pyrmv` to `0.3.5` using [PyPi](https://pypi.org/project/pyrmv) or [Gitea's package index](https://git.end-play.xyz/profitroll/-/packages/pypi/pyrmv) and review if the problem is gone. I assume that RMV did more changes to their API than those found at the moment, but without their HAFAS docs it's hard to quickly figure out which exactly.

profitroll added the

Status

Need More Info

label 2023-11-19 13:32:25 +02:00

marsmathis commented

2023-11-19 16:41:57 +02:00

Okay, I checked everything again. It is as I feared; they changed the way you have to specify the id for a journey. It used to be the JourneyDetailRef found in departures, however apparently that string isn’t supported anymore, at least I haven’t found any example where it yields a result. A friend of mine with whom I tried to understand the API yesterday found a call that works by random chance:

https://www.rmv.de/hapi/journeyDetail?accessId=[yourKey]&format=json&id=1|4166|2|80|18112023

We have no idea why this works or why it yields the data that it does but at least it’s not an error. More investigation would have to be done to figure out the new format.

It’s also possible that this isn’t the case and the JourneyDetailRef still works but we tried for three hours yesterday without any success.

Okay, I checked everything again. It is as I feared; they changed the way you have to specify the id for a journey. It used to be the JourneyDetailRef found in departures, however apparently that string isn’t supported anymore, at least I haven’t found any example where it yields a result. A friend of mine with whom I tried to understand the API yesterday found a call that works by random chance: `https://www.rmv.de/hapi/journeyDetail?accessId=[yourKey]&format=json&id=1|4166|2|80|18112023` We have no idea why this works or why it yields the data that it does but at least it’s not an error. More investigation would have to be done to figure out the new format. It’s also possible that this isn’t the case and the JourneyDetailRef still works but we tried for three hours yesterday without any success.

profitroll commented

2023-11-19 17:18:24 +02:00

True. Gives an error with a random chance.

Moreover, some of my projects relying on pyrmv started giving timeouts and connection resets as well. This is even more terrifying, because I'm not pushing the API or anything, so those are not retelims.

I have requested a new API key. Last time RMV also sent their HAFAS instance docs alongside the API key in email. Hoping for the best.

This API client also needs a rework someday, because it doesn't use asyncio yet and is not scalable at all. Going to take some time to refactor and maybe upgrade pyrmv at some point, yet this situation with API endpoint seems to be a nice call for it, lol.

True. Gives an error with a random chance. Moreover, some of my projects relying on pyrmv started giving timeouts and connection resets as well. This is even more terrifying, because I'm not pushing the API or anything, so those are not retelims. I have requested a new API key. Last time RMV also sent their HAFAS instance docs alongside the API key in email. Hoping for the best. This API client also needs a rework someday, because it doesn't use asyncio yet and is not scalable at all. Going to take some time to refactor and maybe upgrade pyrmv at some point, yet this situation with API endpoint seems to be a nice call for it, lol.

profitroll added

and removed

labels 2023-11-19 17:18:42 +02:00

marsmathis commented

2023-11-19 17:22:09 +02:00

I have the newest HAFAS docs from RMV (well, they sent them two days ago, but I don’t think anything has changed in the meantime) and there’s no info about the journey id situation. It’s definitely not a rate-limit issue and you probably won’t need the new API key because the old one should work just fine. Their API is documented really poorly in general and I think this might be a situation where you’d need to experiment a lot with trial-and-error how the id is actually supposed to be sent.

profitroll commented

2023-11-19 17:30:52 +02:00

Hm. The last version I have seen from them was Server v2.22.1 (2021-11-09). IDs they used before were already marked as deprecated, yet new ones were unsupported at a time.

Anyway, project structure itself needs to be changed quite a lot, so I'm going to start with that and try to investigate the new behavior on the fly.

Their developer support deteriorated quite a bit after DDoS attacks lately, so I'm not expecting much from this adventure at this point. But I'll give it a shot.

Hm. The last version I have seen from them was Server v2.22.1 (2021-11-09). IDs they used before were already marked as deprecated, yet new ones were unsupported at a time. Anyway, project structure itself needs to be changed quite a lot, so I'm going to start with that and try to investigate the new behavior on the fly. Their developer support deteriorated quite a bit after DDoS attacks lately, so I'm not expecting much from this adventure at this point. But I'll give it a shot.

marsmathis commented

2023-11-19 17:33:20 +02:00

I just sent you the latest HAFAS docs on Telegram.

profitroll added a new dependency 2023-11-20 18:40:04 +02:00

#3 Workaround for geo-blocking

profitroll commented

2023-11-24 14:17:49 +02:00

https://www.rmv.de/hapi/journeyDetail?accessId=[yourKey]&format=json&id=1|4166|2|80|18112023

Where did you get that ID from?

I have only seen this kind of JourneyRef in the HAFAS docs. RMV kinda accepts this format, yet does not output it anywhere. Even their rmv.de/hapi/latest uses old Refs.

> `https://www.rmv.de/hapi/journeyDetail?accessId=[yourKey]&format=json&id=1|4166|2|80|18112023` Where did you get that ID from? I have only seen this kind of JourneyRef in the HAFAS docs. RMV kinda accepts this format, yet does not output it anywhere. Even their `rmv.de/hapi/latest` uses old Refs.

marsmathis commented

2023-11-24 15:16:37 +02:00

Yeah, exactly. We only stumbled upon this by accident. We went through the HAFAS docs I sent you and saw this ID format whereupon we started trying to understand the format and then just randomly typed in numbers and at one point this one by accident worked.

profitroll commented

2023-11-25 00:52:20 +02:00

So I used the ID you found as a reference to break it down.

1|4166|2|80|18112023 corresponds to 2|#VN#1#ST#1700765441#PI#0#ZI#4166#TA#2#DA#181123#1S#9994454#1T#802#LS#9994450#LT#807#PU#80#RT#1#CA#DRT#ZE#433.1#ZB# 433.1#PC#9#FR#9994454#FT#802#TO#9994450#TT#807#

My conclusion is:

VN	ZI	TA	PU	DATE (optional)
1	4166	2	80	18112023

I will be able to make a conversion function for it, that's not that hard, actually. But holy crap this looks so unreliable and will definitely break in the future...

That's why I'll alter the "higher-level" interface, but won't touch the raw functions.

So I used the ID you found as a reference to break it down. `1|4166|2|80|18112023` corresponds to `2|#VN#1#ST#1700765441#PI#0#ZI#4166#TA#2#DA#181123#1S#9994454#1T#802#LS#9994450#LT#807#PU#80#RT#1#CA#DRT#ZE#433.1#ZB# 433.1#PC#9#FR#9994454#FT#802#TO#9994450#TT#807#` My conclusion is: | VN | ZI | TA | PU | DATE (optional) | | -- |----- | -- | -- | --------------- | | 1 | 4166 | 2 | 80 | 18112023 | I will be able to make a conversion function for it, that's not that hard, actually. But holy crap this looks so unreliable and will definitely break in the future... That's why I'll alter the "higher-level" interface, but won't touch the raw functions.

profitroll referenced this issue from a commit

2023-11-25 00:53:03 +02:00

Attempt to temporarily fix #2

profitroll commented

2023-11-25 15:40:36 +02:00

Released as v0.4.0-rc.1 (pyrmv 0.4.0rc1 on PyPi).
Please, review this version and report if the problem with the refs persists.

Released as [v0.4.0-rc.1](https://git.end-play.xyz/profitroll/PythonRMV/releases/tag/v0.4.0-rc.1) (pyrmv 0.4.0rc1 on PyPi). Please, review this version and report if the problem with the refs persists.

marsmathis commented

2023-11-27 08:00:47 +02:00

Now the departure board isn’t working.

client.board_departure(origin.id)
Traceback (most recent call last):

  Cell In[81], line 1
    client.board_departure(origin.id)

  File ~/PythonRMV/pyrmv/classes/Client.py:186 in board_departure
    return BoardDeparture(board_raw, self, retrieve_stops=retrieve_stops)

  File ~/PythonRMV/pyrmv/classes/Board.py:88 in __init__
    self.append(LineDeparture(line, client, retrieve_stops=retrieve_stops))

  File ~/PythonRMV/pyrmv/classes/Board.py:38 in __init__
    self.journey = client.journey_detail(data["JourneyDetailRef"]["ref"])

  File ~/PythonRMV/pyrmv/classes/Client.py:357 in journey_detail
    return Journey(journey_raw)

  File ~/PythonRMV/pyrmv/classes/Journey.py:11 in __init__
    self.direction_flag = data["Directions"]["Direction"][0]["flag"]

KeyError: 'flag'

Now the departure board isn’t working. ``` client.board_departure(origin.id) Traceback (most recent call last): Cell In[81], line 1 client.board_departure(origin.id) File ~/PythonRMV/pyrmv/classes/Client.py:186 in board_departure return BoardDeparture(board_raw, self, retrieve_stops=retrieve_stops) File ~/PythonRMV/pyrmv/classes/Board.py:88 in __init__ self.append(LineDeparture(line, client, retrieve_stops=retrieve_stops)) File ~/PythonRMV/pyrmv/classes/Board.py:38 in __init__ self.journey = client.journey_detail(data["JourneyDetailRef"]["ref"]) File ~/PythonRMV/pyrmv/classes/Client.py:357 in journey_detail return Journey(journey_raw) File ~/PythonRMV/pyrmv/classes/Journey.py:11 in __init__ self.direction_flag = data["Directions"]["Direction"][0]["flag"] KeyError: 'flag' ```

profitroll commented

2023-11-27 23:18:29 +02:00

Released as v0.4.0-rc.2 (pyrmv 0.4.0rc2 on PyPi).
Please, review this version and all the breaking changes and report if the problem persists.

Released as [v0.4.0-rc.2](https://git.end-play.xyz/profitroll/PythonRMV/releases/tag/v0.4.0-rc.2) (pyrmv 0.4.0rc2 on PyPi). Please, review this version and all the breaking changes and report if the problem persists.

Sign in to join this conversation.