Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: Support for Microsoft Oauth in Email Channel #6227

Merged
merged 51 commits into from
Jan 16, 2023
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
f3c4fcf
fix:
tejaswinichile Dec 30, 2022
cab4aa3
fix:
tejaswinichile Dec 30, 2022
579bebc
refresh tokens and fetch the mails from ms graph
tejaswinichile Jan 3, 2023
bb5a248
fix: refresh tokens with IMAP; commented conde for the HTTParty
tejaswinichile Jan 4, 2023
3247067
refresh tokens and fetch the mails from ms graph
tejaswinichile Jan 3, 2023
d91d1dc
code cleaning, hash indifferent access fix, updated schedule class an…
tejaswinichile Jan 4, 2023
44a55ac
send email via SMTP
tejaswinichile Jan 5, 2023
c71c602
code sanity changes, addded comments and references
tejaswinichile Jan 5, 2023
7a58a70
rubocop and spec fixes
tejaswinichile Jan 6, 2023
330a36e
Feat: microsoft email channel oauth provider and fetching emails
tejaswinichile Jan 10, 2023
96368e8
minor fixes
tejaswinichile Jan 10, 2023
5f99ffa
feat: code clean up and refactoring
tejaswinichile Jan 11, 2023
48bf5b5
feat: code clean up and refactoring
tejaswinichile Jan 11, 2023
b10ccb6
Remove all the frontend code
pranavrajs Jan 12, 2023
90471d7
Merge branch 'develop' into feat/microsoft-oauth-apis
pranavrajs Jan 12, 2023
7f43ab5
feat: MS reauthorization
tejaswinichile Jan 12, 2023
cb701d6
Merge branch 'feat/microsoft-oauth-apis' of https://github.com/chatwo…
tejaswinichile Jan 12, 2023
3fe0d06
update microsoft oauth concern
tejaswinichile Jan 12, 2023
cd9e93d
PR comments changes
tejaswinichile Jan 12, 2023
d8acd5f
PR comments changes
tejaswinichile Jan 12, 2023
3c23975
Fix: Specs
tejaswinichile Jan 12, 2023
b10d0f3
Merge branch 'feat/microsoft-oauth-apis' of https://github.com/chatwo…
tejaswinichile Jan 12, 2023
8deac0f
fix: rubocop
tejaswinichile Jan 12, 2023
de416ad
fix: rubocop
tejaswinichile Jan 12, 2023
85abe97
Merge branch 'feat/microsoft-oauth-apis' of https://github.com/chatwo…
tejaswinichile Jan 12, 2023
7cc3113
Merge branch 'develop' into feat/microsoft-oauth-apis
tejaswinichile Jan 13, 2023
7521813
fix: remove comman key for redis account id setup
tejaswinichile Jan 13, 2023
d55ff39
fix: remove comman key for redis account id setup
tejaswinichile Jan 13, 2023
179fb37
fix: remove comman key for redis account id setup
tejaswinichile Jan 13, 2023
8c668f6
Merge branch 'feat/microsoft-oauth-apis' of https://github.com/chatwo…
tejaswinichile Jan 13, 2023
d5a8e05
fix: PR comments resolved
tejaswinichile Jan 13, 2023
f0bcdda
fix: moved refersh_ms_oauth_token job to service
tejaswinichile Jan 13, 2023
4b28672
chore: refactor the token service
sojan-official Jan 13, 2023
d53b077
chore: rubocop fixes
sojan-official Jan 13, 2023
2708fc2
spces for callbacks controlelr
tejaswinichile Jan 13, 2023
5d712dd
spces for callbacks controlelr
tejaswinichile Jan 13, 2023
526b83a
Merge branch 'feat/microsoft-oauth-apis' of https://github.com/chatwo…
tejaswinichile Jan 13, 2023
9d941de
Merge branch 'develop' into feat/microsoft-oauth-apis
tejaswinichile Jan 13, 2023
799f46b
fix: failing specs
tejaswinichile Jan 16, 2023
5eab1de
Merge branch 'develop' into feat/microsoft-oauth-apis
tejaswinichile Jan 16, 2023
f18d7a5
chore: remove unneccesary mocks
sojan-official Jan 16, 2023
8e55349
Merge branch 'develop' into feat/microsoft-oauth-apis
sojan-official Jan 16, 2023
908c409
chore: callback controller specs
sojan-official Jan 16, 2023
6a9f78c
chore: remove unwanted code
sojan-official Jan 16, 2023
6adebd1
chore: spec for model
sojan-official Jan 16, 2023
80d977b
chore: rubocop
sojan-official Jan 16, 2023
7da09f6
chore: more specs
sojan-official Jan 16, 2023
cae923b
chore: refactor
sojan-official Jan 16, 2023
ba09939
chore: refactor
sojan-official Jan 16, 2023
10cf72c
chore: fix specs
sojan-official Jan 16, 2023
759fecf
Merge branch 'develop' into feat/microsoft-oauth-apis
sojan-official Jan 16, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ REDIS_SENTINELS=
REDIS_SENTINEL_MASTER_NAME=

# By default Chatwoot will pass REDIS_PASSWORD as the password value for sentinels
# Use the following environment variable to customize passwords for sentinels.
# Use the following environment variable to customize passwords for sentinels.
# Use empty string if sentinels are configured with out passwords
# REDIS_SENTINEL_PASSWORD=

Expand All @@ -45,7 +45,7 @@ REDIS_SENTINEL_MASTER_NAME=
# REDIS_OPENSSL_VERIFY_MODE=none

# Postgres Database config variables
# You can leave POSTGRES_DATABASE blank. The default name of
# You can leave POSTGRES_DATABASE blank. The default name of
# the database in the production environment is chatwoot_production
# POSTGRES_DATABASE=
POSTGRES_HOST=postgres
Expand Down Expand Up @@ -213,3 +213,7 @@ STRIPE_WEBHOOK_SECRET=
# Set to true if you want to upload files to cloud storage using the signed url
# Make sure to follow https://edgeguides.rubyonrails.org/active_storage_overview.html#cross-origin-resource-sharing-cors-configuration on the cloud storage after setting this to true.
DIRECT_UPLOADS_ENABLED=

#MS OAUTH creds
AZURE_APP_ID=
AZURE_APP_SECRET=
4 changes: 4 additions & 0 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@ gem 'json_schemer'
gem 'rack-attack'
# a utility tool for streaming, flexible and safe downloading of remote files
gem 'down', '~> 5.0'
# authentication type to fetch and send mail over oauth2.0
gem 'gmail_xoauth'

##-- for active storage --##
gem 'aws-sdk-s3', require: false
Expand Down Expand Up @@ -186,3 +188,5 @@ group :development, :test do
gem 'spring'
gem 'spring-watcher-listen'
end
# worked with microsoft refresh token
gem 'omniauth-oauth2'
24 changes: 24 additions & 0 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -249,6 +249,8 @@ GEM
gli (2.21.0)
globalid (1.0.0)
activesupport (>= 5.0)
gmail_xoauth (0.4.2)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tejaswinichile net::imap latest versions support xoauth automatically. We can upgrade the package. No need to do it in this PR. We can take it as a separate task.

oauth (>= 0.3.6)
google-apis-core (0.7.0)
addressable (~> 2.5, >= 2.5.1)
googleauth (>= 0.16.2, < 2.a)
Expand Down Expand Up @@ -437,6 +439,20 @@ GEM
nokogiri (1.13.10-x86_64-linux)
racc (~> 1.4)
oauth (0.5.10)
oauth2 (2.0.9)
faraday (>= 0.17.3, < 3.0)
jwt (>= 1.0, < 3.0)
multi_xml (~> 0.5)
rack (>= 1.2, < 4)
snaky_hash (~> 2.0)
version_gem (~> 1.1)
omniauth (2.1.0)
hashie (>= 3.4.6)
rack (>= 2.2.3)
rack-protection
omniauth-oauth2 (1.8.0)
oauth2 (>= 1.4, < 3)
omniauth (~> 2.0)
orm_adapter (0.5.0)
os (1.1.4)
parallel (1.22.1)
Expand Down Expand Up @@ -465,6 +481,8 @@ GEM
rack (>= 1.0, < 3)
rack-cors (1.1.1)
rack (>= 2.0.0)
rack-protection (3.0.5)
rack
rack-proxy (0.7.2)
rack
rack-test (2.0.2)
Expand Down Expand Up @@ -620,6 +638,9 @@ GEM
gli
hashie
websocket-driver
snaky_hash (2.0.1)
hashie
version_gem (~> 1.1, >= 1.1.1)
spring (2.1.1)
spring-watcher-listen (2.0.1)
listen (>= 2.7, < 4.0)
Expand Down Expand Up @@ -663,6 +684,7 @@ GEM
valid_email2 (4.0.3)
activemodel (>= 3.2)
mail (~> 2.5)
version_gem (1.1.1)
warden (1.2.9)
rack (>= 2.0.9)
web-console (4.2.0)
Expand Down Expand Up @@ -735,6 +757,7 @@ DEPENDENCIES
flag_shih_tzu
foreman
geocoder
gmail_xoauth
google-cloud-dialogflow
google-cloud-storage
groupdate
Expand All @@ -756,6 +779,7 @@ DEPENDENCIES
maxminddb
mock_redis
newrelic_rpm
omniauth-oauth2
pg
pg_search
procore-sift
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
class Api::V1::Accounts::Microsoft::AuthorizationsController < Api::V1::Accounts::BaseController
include MicrosoftConcern
before_action :check_authorization

def create
email = params[:authorization][:email]
redirect_url = microsoft_client.auth_code.authorize_url({
redirect_uri: "#{base_url}/microsoft/callback",
scope: 'offline_access https://outlook.office.com/IMAP.AccessAsUser.All
https://outlook.office.com/SMTP.Send openid',
prompt: 'consent'
})
if redirect_url
::Redis::Alfred.setex(email, Current.account.id, 5.minutes)
render json: { success: true, url: redirect_url }
else
render json: { success: false }, status: :unprocessable_entity
end
end

private

def check_authorization
raise Pundit::NotAuthorizedError unless Current.account_user.administrator?
end
end
22 changes: 22 additions & 0 deletions app/controllers/concerns/microsoft_concern.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
module MicrosoftConcern
extend ActiveSupport::Concern

def microsoft_client
::OAuth2::Client.new(ENV.fetch('AZURE_APP_ID', nil), ENV.fetch('AZURE_APP_SECRET', nil),
{
site: 'https://login.microsoftonline.com',
authorize_url: 'https://login.microsoftonline.com/common/oauth2/v2.0/authorize',
token_url: 'https://login.microsoftonline.com/common/oauth2/v2.0/token'
})
end

private

def parsed_body
@parsed_body ||= Rack::Utils.parse_nested_query(@response.raw_response.body)
end

def base_url
ENV.fetch('FRONTEND_URL', 'http://localhost:3000')
end
end
73 changes: 73 additions & 0 deletions app/controllers/microsoft/callbacks_controller.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
class Microsoft::CallbacksController < ApplicationController
include MicrosoftConcern

def show
@response = microsoft_client.auth_code.get_token(
oauth_code,
redirect_uri: "#{base_url}/microsoft/callback"
)

ActiveRecord::Base.transaction do
inbox = find_or_create_inbox
::Redis::Alfred.delete(users_data['email'])
redirect_to app_microsoft_inbox_agents_url(account_id: account.id, inbox_id: inbox.id)
rescue StandardError => e
Rails.logger.error e
redirect_to microsoft_app_redirect_url
end
end

private

def oauth_code
params[:code]
end

def users_data
decoded_token = JWT.decode parsed_body[:id_token], nil, false
decoded_token[0]
end

def parsed_body
@parsed_body ||= @response.response.parsed
end

def account_id
::Redis::Alfred.get(users_data['email'])
end

def account
@account ||= Account.find(account_id)
end

def microsoft_app_redirect_url
app_new_microsoft_inbox_url(account_id: account.id)
end

def find_or_create_inbox
channel_email = create_imap_email_channel

return channel_email.inbox if channel_email.inbox.presence

account.inboxes.create_or_find_by!(
account: account,
channel: channel_email,
name: users_data['name']
)
end

def create_imap_email_channel
channel_email = Channel::Email.find_or_create_by!(email: users_data['email'], account: account)
channel_email.update!({
imap_login: users_data['email'], imap_address: 'outlook.office365.com',
imap_port: '993', imap_enabled: true,
provider: 'microsoft',
provider_config: {
access_token: parsed_body['access_token'],
refresh_token: parsed_body['refresh_token'],
expires_on: (Time.current.utc + 1.hour).to_s
}
})
channel_email
end
end
8 changes: 8 additions & 0 deletions app/javascript/dashboard/i18n/locale/en/inboxMgmt.json
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,10 @@
"ENABLE": "Create conversations from mentioned Tweets"
}
},
"MICROSOFT": {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be a part of the frontend PR ?

"HELP": "To add your Microsoft account as a channel, you need to authenticate your Microsoft account by clicking on 'Sign in with Microsoft' ",
"ERROR_MESSAGE": "There was an error connecting to Microsoft, please try again"
},
"WEBSITE_CHANNEL": {
"TITLE": "Website channel",
"DESC": "Create a channel for your website and start supporting your customers via our website widget.",
Expand Down Expand Up @@ -548,6 +552,10 @@
},
"ENABLE_SSL": "Enable SSL"
},
"MICROSOFT": {
"TITLE": "Microsoft",
"SUBTITLE": "Reauthorize your MICROSOFT account"
},
"SMTP": {
"TITLE": "SMTP",
"SUBTITLE": "Set your SMTP details",
Expand Down
47 changes: 47 additions & 0 deletions app/jobs/inboxes/fetch_imap_emails_job.rb
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,10 @@ class Inboxes::FetchImapEmailsJob < ApplicationJob
def perform(channel)
return unless should_fetch_email?(channel)

# fetching email for microsoft provider
fetch_mail_for_ms_provider(channel)
fetch_mail_for_channel(channel)

# clearing old failures like timeouts since the mail is now successfully processed
channel.reauthorized!
rescue *ExceptionList::IMAP_EXCEPTIONS
Expand All @@ -24,6 +27,8 @@ def should_fetch_email?(channel)
end

def fetch_mail_for_channel(channel)
return if channel.microsoft?

# TODO: rather than setting this as default method for all mail objects, lets if can do new mail object
# using Mail.retriever_method.new(params)
Mail.defaults do
Expand All @@ -41,9 +46,51 @@ def fetch_mail_for_channel(channel)
end
end

def fetch_mail_for_ms_provider(channel)
return unless channel.microsoft? && channel.provider_config[:access_token].blank?

access_token = valid_access_token channel

return unless access_token

imap = imap_authenticate(channel, access_token)

process_mails(imap, channel)
end

def process_mails(imap, channel)
imap.search(['BEFORE', tomorrow, 'SINCE', yesterday]).each do |message_id|
inbound_mail = Mail.read_from_string imap.fetch(message_id, 'RFC822')[0].attr['RFC822']

next if channel.inbox.messages.find_by(source_id: inbound_mail.message_id).present?

process_mail(inbound_mail, channel)
end
end

def imap_authenticate(channel, access_token)
imap = Net::IMAP.new(channel.imap_address, channel.imap_port, true)
imap.authenticate('XOAUTH2', channel.imap_login, access_token)
imap.select('INBOX')
imap
end

def yesterday
(Time.zone.today - 1).strftime('%d-%b-%Y')
end

def tomorrow
(Time.zone.today + 1).strftime('%d-%b-%Y')
end

def process_mail(inbound_mail, channel)
Imap::ImapMailbox.new.process(inbound_mail, channel)
rescue StandardError => e
ChatwootExceptionTracker.new(e, account: channel.account).capture_exception
end

# Making sure the access token is valid for microsoft provider
def valid_access_token(channel)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will be doing this step every time when we fetch the mail right ? ( in 1 minute ). In case we won't be needing to run the job separately in a schedule ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yes! you are right :D.. This is happening every 5 minute to check if the token is still valid and if not it will request for new token.

Removing the refresh token job.

Microsoft::RefreshOauthTokenService.new(channel: channel).access_token
end
end
28 changes: 26 additions & 2 deletions app/mailers/conversation_reply_mailer_helper.rb
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,33 @@ def prepare_mail(cc_bcc_enabled)
@options[:cc] = cc_bcc_emails[0]
@options[:bcc] = cc_bcc_emails[1]
end

ms_smtp_settings
set_delivery_method

mail(@options)
end

private

def ms_smtp_settings
return unless @inbox.email? && @channel.imap_enabled && @inbox.channel.provider == 'microsoft'

smtp_settings = {
address: 'smtp.office365.com',
port: 587,
user_name: @channel.imap_login,
password: @channel.provider_config['access_token'],
domain: 'smtp.office365.com',
tls: false,
enable_starttls_auto: true,
openssl_verify_mode: 'none',
authentication: 'xoauth2'
}

@options[:delivery_method] = :smtp
@options[:delivery_method_options] = smtp_settings
end

def set_delivery_method
return unless @inbox.inbox_type == 'Email' && @channel.smtp_enabled

Expand Down Expand Up @@ -47,8 +67,12 @@ def email_imap_enabled
@inbox.inbox_type == 'Email' && @channel.imap_enabled
end

def email_microsoft_auth_enabled
@inbox.inbox_type == 'Email' && @channel.provider == 'microsoft'
end

def email_from
email_smtp_enabled ? @channel.email : from_email_with_name
email_microsoft_auth_enabled || email_smtp_enabled ? @channel.email : from_email_with_name
end

def email_reply_to
Expand Down
Loading